[clang] [llvm] [mlir] [polly] [IR] change i128 datalayout default to match clang (PR #214518)
Simeon David Schaub via llvm-commits
llvm-commits at lists.llvm.org
Sat Aug 8 00:03:13 PDT 2026
https://github.com/simeonschaub updated https://github.com/llvm/llvm-project/pull/214518
>From a1fc128bacc883ca4695affca65bf93ad26f5d63 Mon Sep 17 00:00:00 2001
From: Simeon David Schaub <simeon at schaub.rocks>
Date: Thu, 6 Aug 2026 15:48:16 +0000
Subject: [PATCH 1/3] [IR] Give i128 a default ABI alignment of 16 in the data
layout
Data layout strings without an explicit i128 entry used to fall back to
the alignment of the next smaller integer entry, typically i64:64. Clang
however defaults Int128Align to 128 bits, so on every target that didn't
spell out an i128 entry, the struct layouts computed by LLVM disagreed
with the ABI implemented by the frontend. On AMDGPU this caused
miscompiles of kernels taking i128 arguments, since the kernarg segment
is laid out with the LLVM rules (see
https://github.com/JuliaGPU/AMDGPU.jl/issues/1002).
Fix this by adding i128:128:128 to the default integer specifications,
matching the Clang default. SystemZ is the only target whose ABI aligns
__int128 to only 8 bytes, so its data layout now spells out i128:64
explicitly, and data layouts of existing SystemZ IR are upgraded
accordingly. All other targets either already declare i128:128 or
inherit the new default.
Assisted-by: Claude Code (claude-fable-5)
---
clang/test/CodeGen/target-data.c | 6 +-
llvm/docs/LangRef.md | 4 +-
llvm/docs/ReleaseNotes.md | 8 ++
llvm/lib/IR/AutoUpgrade.cpp | 10 ++-
llvm/lib/IR/DataLayout.cpp | 9 +-
llvm/lib/TargetParser/TargetDataLayout.cpp | 4 +
llvm/test/Bitcode/upgrade-datalayout6.ll | 9 ++
.../CodeGen/AMDGPU/kernarg-i128-alignment.ll | 83 +++++++++++++++++++
.../Bitcode/DataLayoutUpgradeTest.cpp | 11 ++-
9 files changed, 132 insertions(+), 12 deletions(-)
create mode 100644 llvm/test/Bitcode/upgrade-datalayout6.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/kernarg-i128-alignment.ll
diff --git a/clang/test/CodeGen/target-data.c b/clang/test/CodeGen/target-data.c
index f2a09a40ee685..2373eefa2e4b2 100644
--- a/clang/test/CodeGen/target-data.c
+++ b/clang/test/CodeGen/target-data.c
@@ -207,11 +207,11 @@
// RUN: FileCheck %s -check-prefix=SYSTEMZ
// RUN: %clang_cc1 -triple s390x-unknown -target-cpu z13 -target-feature +soft-float -o - -emit-llvm %s | \
// RUN: FileCheck %s -check-prefix=SYSTEMZ
-// SYSTEMZ: target datalayout = "E-S64-m:e-i1:8:16-i8:8:16-i64:64-f128:64-v128:64-a:8:16-n32:64"
+// SYSTEMZ: target datalayout = "E-S64-m:e-i1:8:16-i8:8:16-i64:64-i128:64-f128:64-v128:64-a:8:16-n32:64"
// RUN: %clang_cc1 -triple s390x-unknown -target-cpu z13 -o - -emit-llvm %s | \
// RUN: FileCheck %s -check-prefix=SYSTEMZ-VECTOR
-// SYSTEMZ-VECTOR: target datalayout = "E-S64-m:e-i1:8:16-i8:8:16-i64:64-f128:64-v128:64-a:8:16-n32:64"
+// SYSTEMZ-VECTOR: target datalayout = "E-S64-m:e-i1:8:16-i8:8:16-i64:64-i128:64-f128:64-v128:64-a:8:16-n32:64"
// RUN: %clang_cc1 -triple s390x-none-zos -target-cpu z10 -o - -emit-llvm %s | \
// RUN: FileCheck %s -check-prefix=ZOS
@@ -219,7 +219,7 @@
// RUN: FileCheck %s -check-prefix=ZOS
// RUN: %clang_cc1 -triple s390x-none-zos -target-cpu z13 -o - -emit-llvm %s | \
// RUN: FileCheck %s -check-prefix=ZOS
-// ZOS: target datalayout = "E-S64-m:l-p1:32:32-i1:8:16-i8:8:16-i64:64-f128:64-v128:64-a:8:16-n32:64"
+// ZOS: target datalayout = "E-S64-m:l-p1:32:32-i1:8:16-i8:8:16-i64:64-i128:64-f128:64-v128:64-a:8:16-n32:64"
// RUN: %clang_cc1 -triple msp430-unknown -o - -emit-llvm %s | \
// RUN: FileCheck %s -check-prefix=MSP430
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index 6a3194271b838..02de32e247381 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -3649,6 +3649,7 @@ specifications are given in this list:
- `i32:32:32` - i32 is 32-bit aligned
- `i64:32:64` - i64 has ABI alignment of 32-bits but preferred
alignment of 64-bits
+- `i128:128:128` - i128 is 128-bit aligned
- `f16:16:16` - half is 16-bit aligned
- `f32:32:32` - float is 32-bit aligned
- `f64:64:64` - double is 64-bit aligned
@@ -3668,7 +3669,8 @@ following rules:
the bitwidth then the largest integer type is used. For example,
given the default specifications above, the i7 type will use the
alignment of i8 (next largest) while both i65 and i256 will use the
- alignment of i64 (largest specified).
+ alignment of i128 (i65 because it is the next largest, i256 because
+ i128 is the largest specified).
The function of the data layout string may not be what you expect.
Notably, this is not a specification from the frontend of what alignment
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index f8fce847b62f3..58d77dac0421b 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -52,6 +52,14 @@ Makes programs 10x faster by doing Special New Thing.
### Changes to the LLVM IR
+* The default ABI and preferred alignment of `i128` in the data layout is now
+ 128 bits, matching the `__int128` ABI implemented by Clang. Previously, data
+ layout strings without an explicit `i128` entry fell back to the alignment of
+ the next smaller specified integer type, typically `i64`. Targets whose ABI
+ aligns `i128` to fewer than 128 bits (currently only SystemZ) now spell this
+ out explicitly in their data layout string, and data layouts of existing
+ SystemZ IR are upgraded accordingly.
+
### Changes to LLVM infrastructure
### Changes to building LLVM
diff --git a/llvm/lib/IR/AutoUpgrade.cpp b/llvm/lib/IR/AutoUpgrade.cpp
index d2216e4072337..bbc524afc218c 100644
--- a/llvm/lib/IR/AutoUpgrade.cpp
+++ b/llvm/lib/IR/AutoUpgrade.cpp
@@ -7270,10 +7270,16 @@ std::string llvm::UpgradeDataLayoutString(StringRef DL, StringRef TT) {
}
if (T.isSystemZ() && !DL.empty()) {
+ Res = DL.str();
+ // i128 used to be aligned only to 64 bits by way of the i64:64 entry.
+ // Now that the default i128 alignment is 128 bits, this needs to be
+ // spelled out explicitly.
+ if (!DL.contains("-i128") && !DL.starts_with("i128"))
+ Res.append("-i128:64");
// Make sure the stack alignment is present.
if (!DL.contains("-S64"))
- return "E-S64" + DL.drop_front(1).str();
- return DL.str();
+ Res = "E-S64" + StringRef(Res).drop_front(1).str();
+ return Res;
}
auto AddPtr32Ptr64AddrSpaces = [&DL, &Res]() {
diff --git a/llvm/lib/IR/DataLayout.cpp b/llvm/lib/IR/DataLayout.cpp
index 82b33887b81f2..a480768947d34 100644
--- a/llvm/lib/IR/DataLayout.cpp
+++ b/llvm/lib/IR/DataLayout.cpp
@@ -178,10 +178,11 @@ struct LessPointerAddrSpace {
// Default primitive type specifications.
// NOTE: These arrays must be sorted by type bit width.
constexpr DataLayout::PrimitiveSpec DefaultIntSpecs[] = {
- {8, Align::Constant<1>(), Align::Constant<1>()}, // i8:8:8
- {16, Align::Constant<2>(), Align::Constant<2>()}, // i16:16:16
- {32, Align::Constant<4>(), Align::Constant<4>()}, // i32:32:32
- {64, Align::Constant<4>(), Align::Constant<8>()}, // i64:32:64
+ {8, Align::Constant<1>(), Align::Constant<1>()}, // i8:8:8
+ {16, Align::Constant<2>(), Align::Constant<2>()}, // i16:16:16
+ {32, Align::Constant<4>(), Align::Constant<4>()}, // i32:32:32
+ {64, Align::Constant<4>(), Align::Constant<8>()}, // i64:32:64
+ {128, Align::Constant<16>(), Align::Constant<16>()}, // i128:128:128
};
constexpr DataLayout::PrimitiveSpec DefaultFloatSpecs[] = {
{16, Align::Constant<2>(), Align::Constant<2>()}, // f16:16:16
diff --git a/llvm/lib/TargetParser/TargetDataLayout.cpp b/llvm/lib/TargetParser/TargetDataLayout.cpp
index 8b6f46642e4fa..b0f0348ce4e1d 100644
--- a/llvm/lib/TargetParser/TargetDataLayout.cpp
+++ b/llvm/lib/TargetParser/TargetDataLayout.cpp
@@ -389,6 +389,10 @@ static std::string computeSystemZDataLayout(const Triple &TT) {
// 64-bit integers are naturally aligned.
Ret += "-i64:64";
+ // 128-bit integers are aligned only to 64 bits, unlike the 128-bit default
+ // of the data layout.
+ Ret += "-i128:64";
+
// 128-bit floats are aligned only to 64 bits.
Ret += "-f128:64";
diff --git a/llvm/test/Bitcode/upgrade-datalayout6.ll b/llvm/test/Bitcode/upgrade-datalayout6.ll
new file mode 100644
index 0000000000000..8bbcfdcd77d22
--- /dev/null
+++ b/llvm/test/Bitcode/upgrade-datalayout6.ll
@@ -0,0 +1,9 @@
+; Test that an explicit i128:64 entry is added to SystemZ data layouts, which
+; predate the change of the default i128 alignment to 128 bits.
+;
+; RUN: llvm-as %s -o - | llvm-dis - | FileCheck %s
+
+target datalayout = "E-m:e-i1:8:16-i8:8:16-i64:64-f128:64-v128:64-a:8:16-n32:64-S64"
+target triple = "s390x-unknown-linux-gnu"
+
+; CHECK: target datalayout = "E-m:e-i1:8:16-i8:8:16-i64:64-f128:64-v128:64-a:8:16-n32:64-S64-i128:64"
diff --git a/llvm/test/CodeGen/AMDGPU/kernarg-i128-alignment.ll b/llvm/test/CodeGen/AMDGPU/kernarg-i128-alignment.ll
new file mode 100644
index 0000000000000..dacbaedcaa37f
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/kernarg-i128-alignment.ll
@@ -0,0 +1,83 @@
+; RUN: llc -mtriple=amdgpu11.00-amd-amdhsa < %s | FileCheck %s
+
+; The data layout has to give i128 an ABI alignment of 16, matching the ABI
+; implemented by Clang, whose AMDGPUTargetInfo leaves Int128Align at its
+; 128-bit default. This now comes from the default i128:128 entry of the data
+; layout; it used to be inherited from the i64:64 entry, and the layout LLVM
+; computed for an aggregate would then disagree with the one a frontend used
+; when it emitted the field offsets.
+;
+; For kernel arguments that disagreement is an ABI break rather than a missed
+; optimization: the kernarg slot is sized from the data layout, so the tail of
+; the argument is never copied into the kernarg segment and loads of it run off
+; the end of the segment.
+
+; struct S { i64 a; i128 b; }: ABI align 16, size 32, offsetof(b) == 16.
+; The byref slot must be 32 bytes, not 24, and `b` must be loaded from 0x20
+; (kernarg base 16 plus a field offset of 16) which stays inside the segment.
+; CHECK-LABEL: {{^}}kernarg_i128:
+; CHECK: s_load_b128 s[{{[0-9]+:[0-9]+}}], s[{{[0-9]+:[0-9]+}}], 0x20
+
+; An i128 following a smaller member is padded out to offset 16 rather than
+; packed at offset 8.
+; CHECK-LABEL: {{^}}kernarg_i128_after_i8:
+; CHECK: s_load_b128 s[{{[0-9]+:[0-9]+}}], s[{{[0-9]+:[0-9]+}}], 0x20
+
+; A bare i128 kernel argument is 16-byte aligned in the kernarg segment, so it
+; starts at 16 (not 8) and the argument after it at 32 (not 24).
+; CHECK-LABEL: {{^}}kernarg_i128_scalar:
+; CHECK: s_load_b128 s[{{[0-9]+:[0-9]+}}], s[{{[0-9]+:[0-9]+}}], 0x10
+
+; The kernel metadata is emitted once, after every function, so the per-kernel
+; argument offsets are checked here in order rather than under each label.
+;
+; .kernarg_segment_size is deliberately not checked: it also covers the 256
+; bytes of hidden implicit arguments, which is noise for what this test pins
+; down. The per-argument .offset/.size entries and .kernarg_segment_align are
+; the layout facts that matter.
+; CHECK: .amdgpu_metadata
+
+; CHECK: .name: s
+; CHECK-NEXT: .offset: 16
+; CHECK-NEXT: .size: 32
+; CHECK: .kernarg_segment_align: 16
+; CHECK: .name: kernarg_i128
+;
+; CHECK: .name: s
+; CHECK-NEXT: .offset: 16
+; CHECK-NEXT: .size: 32
+; CHECK: .name: kernarg_i128_after_i8
+;
+; CHECK: .name: a
+; CHECK-NEXT: .offset: 16
+; CHECK-NEXT: .size: 16
+; CHECK: .name: b
+; CHECK-NEXT: .offset: 32
+; CHECK-NEXT: .size: 8
+; CHECK: .name: kernarg_i128_scalar
+
+define amdgpu_kernel void @kernarg_i128(ptr addrspace(1) %out,
+ ptr addrspace(4) byref({ i64, i128 }) align 16 %s) {
+ %pb = getelementptr inbounds i8, ptr addrspace(4) %s, i64 16
+ %b = load i128, ptr addrspace(4) %pb, align 16
+ store i128 %b, ptr addrspace(1) %out, align 16
+ ret void
+}
+
+define amdgpu_kernel void @kernarg_i128_after_i8(ptr addrspace(1) %out,
+ ptr addrspace(4) byref({ i8, i128 }) align 16 %s) {
+ %pb = getelementptr inbounds i8, ptr addrspace(4) %s, i64 16
+ %b = load i128, ptr addrspace(4) %pb, align 16
+ store i128 %b, ptr addrspace(1) %out, align 16
+ ret void
+}
+
+define amdgpu_kernel void @kernarg_i128_scalar(ptr addrspace(1) %out, i128 %a, i64 %b) {
+ %ext = zext i64 %b to i128
+ %sum = add i128 %a, %ext
+ store i128 %sum, ptr addrspace(1) %out, align 16
+ ret void
+}
+
+!llvm.module.flags = !{!0}
+!0 = !{i32 1, !"amdhsa_code_object_version", i32 500}
diff --git a/llvm/unittests/Bitcode/DataLayoutUpgradeTest.cpp b/llvm/unittests/Bitcode/DataLayoutUpgradeTest.cpp
index a082adbf6565e..d9cf400e799d4 100644
--- a/llvm/unittests/Bitcode/DataLayoutUpgradeTest.cpp
+++ b/llvm/unittests/Bitcode/DataLayoutUpgradeTest.cpp
@@ -68,11 +68,18 @@ TEST(DataLayoutUpgradeTest, ValidDataLayoutUpgrade) {
"1024-v2048:2048-n32:64-S32-A5-G1-ni:7:8:9-p7:160:256:256:32-p8:128:128:"
"128:48-p9:192:256:256:32");
- // Check that SystemZ adds -S64 if needed.
+ // Check that SystemZ adds -S64 and -i128:64 if needed.
EXPECT_EQ(UpgradeDataLayoutString(
"E-m:e-i1:8:16-i8:8:16-i64:64-f128:64-v128:64-a:8:16-n32:64",
"systemz"),
- "E-S64-m:e-i1:8:16-i8:8:16-i64:64-f128:64-v128:64-a:8:16-n32:64");
+ "E-S64-m:e-i1:8:16-i8:8:16-i64:64-f128:64-v128:64-a:8:16-n32:64-"
+ "i128:64");
+ // but doesn't add -i128:64 if it is already present.
+ EXPECT_EQ(UpgradeDataLayoutString("E-m:e-i1:8:16-i8:8:16-i64:64-i128:64-"
+ "f128:64-v128:64-a:8:16-n32:64-S64",
+ "systemz"),
+ "E-m:e-i1:8:16-i8:8:16-i64:64-i128:64-f128:64-v128:64-a:8:16-n32:"
+ "64-S64");
// Check that RISCV64 upgrades -n64 to -n32:64.
EXPECT_EQ(UpgradeDataLayoutString("e-m:e-p:64:64-i64:64-i128:128-n64-S128",
>From 604263cb47e01425d3d4f645ab585ab2d4f17ead Mon Sep 17 00:00:00 2001
From: Simeon David Schaub <simeon at schaub.rocks>
Date: Thu, 6 Aug 2026 15:48:17 +0000
Subject: [PATCH 2/3] [Tests] Update test expectations for the i128 default
alignment change
Mostly mechanical regeneration with the UTC scripts: wide integer
(i65 and up) allocas, loads and stores in modules without an explicit
i128 data layout entry now get an ABI alignment of 16, and AMDGPU
kernarg offsets after an i128 argument shift to the next 16-byte slot.
Notable non-mechanical updates:
- llubi/loadstore_{be,le}.ll: the alignment-less b256 load would now
imply an alignment of 16 and trap on the 8-aligned alloca, so it gets
an explicit align 8.
- InstCombine/select.ll (test86): the i128 loads are no longer
speculatable across the select (required alignment 16 exceeds the
8-aligned allocas), so the select of pointers is kept, matching the
existing behavior on targets that declare i128:128 explicitly.
- OpenMPIRBuilderTest.CreateTask: the {ptr, i128} shareds struct is now
32 bytes instead of 24.
Assisted-by: Claude Code (claude-fable-5)
---
clang/test/CodeGen/LoongArch/bitint.c | 6 +-
clang/test/CodeGenHIP/printf_nonhostcall.cpp | 4 +-
.../CostModel/AMDGPU/load-to-trunc.ll | 4 +-
.../Analysis/CostModel/ARM/load-to-trunc.ll | 4 +-
.../LoopAccessAnalysis/depend_diff_types.ll | 2 +-
.../AMDGPU/GlobalISel/function-returns.ll | 12 +-
.../GlobalISel/irtranslator-function-args.ll | 12 +-
.../regbankselect-amdgcn.s.buffer.load.ll | 56 +--
.../CodeGen/AMDGPU/GlobalISel/trunc-brc.ll | 106 ++---
llvm/test/CodeGen/AMDGPU/add_i128.ll | 2 +-
.../amdgpu-attributor-min-agpr-alloc.ll | 2 +-
llvm/test/CodeGen/AMDGPU/ctpop64.ll | 69 ++--
llvm/test/CodeGen/AMDGPU/kernel-args.ll | 61 +--
.../AMDGPU/kernel-argument-dag-lowering.ll | 15 +-
.../lower-buffer-fat-pointers-constants.ll | 2 +-
...ffer-fat-pointers-contents-legalization.ll | 24 +-
llvm/test/CodeGen/AMDGPU/mul.ll | 64 +--
llvm/test/CodeGen/AMDGPU/opencl-printf.ll | 8 +-
.../preload-implicit-kernargs-IR-lowering.ll | 4 +-
.../AMDGPU/preload-implicit-kernargs.ll | 17 +-
llvm/test/CodeGen/AMDGPU/store-weird-sizes.ll | 20 +-
.../ARM/cmse-harden-entry-arguments.ll | 62 ++-
llvm/test/CodeGen/ARM/emit-big-cst.ll | 4 +-
llvm/test/CodeGen/ARM/llvm.sincos.ll | 50 ++-
llvm/test/CodeGen/ARM/store-postinc.ll | 203 +++++-----
llvm/test/CodeGen/ARM/store-preinc.ll | 203 +++++-----
.../ARM/umulo-128-legalisation-lowering.ll | 328 +++++++--------
.../CodeGen/ARM/unaligned_load_store_aeabi.ll | 40 +-
llvm/test/CodeGen/AVR/llvm.sincos.ll | 80 ++--
llvm/test/CodeGen/Hexagon/llvm.exp10.ll | 36 +-
llvm/test/CodeGen/Hexagon/llvm.sincos.ll | 198 ++++-----
llvm/test/CodeGen/LoongArch/llvm.exp10.ll | 60 +--
llvm/test/CodeGen/LoongArch/llvm.sincos.ll | 120 +++---
.../LoongArch/musttail-indirect-args.ll | 30 +-
llvm/test/CodeGen/LoongArch/soft-fp-to-int.ll | 10 +-
llvm/test/CodeGen/MSP430/llvm.exp10.ll | 56 +--
.../CodeGen/Mips/GlobalISel/llvm-ir/add.ll | 22 +-
.../CodeGen/Mips/GlobalISel/llvm-ir/mul.ll | 22 +-
.../CodeGen/Mips/GlobalISel/llvm-ir/sub.ll | 22 +-
llvm/test/CodeGen/Mips/llvm-ir/add-dsp.ll | 46 ++-
llvm/test/CodeGen/Mips/llvm-ir/add.ll | 50 +--
llvm/test/CodeGen/Mips/llvm-ir/and.ll | 114 +++++-
llvm/test/CodeGen/Mips/llvm-ir/ashr.ll | 120 ++++--
llvm/test/CodeGen/Mips/llvm-ir/lshr.ll | 120 ++++--
llvm/test/CodeGen/Mips/llvm-ir/not.ll | 71 +++-
llvm/test/CodeGen/Mips/llvm-ir/or.ll | 66 ++-
llvm/test/CodeGen/Mips/llvm-ir/sdiv.ll | 142 ++++---
llvm/test/CodeGen/Mips/llvm-ir/shl.ll | 134 +++++--
llvm/test/CodeGen/Mips/llvm-ir/srem.ll | 142 ++++---
llvm/test/CodeGen/Mips/llvm-ir/sub.ll | 34 +-
llvm/test/CodeGen/Mips/llvm-ir/udiv.ll | 142 ++++---
llvm/test/CodeGen/Mips/llvm-ir/urem.ll | 142 ++++---
llvm/test/CodeGen/Mips/llvm-ir/xor.ll | 115 +++++-
llvm/test/CodeGen/Mips/llvm.sincos.ll | 220 +++++-----
llvm/test/CodeGen/PowerPC/all-atomics.ll | 146 +++----
llvm/test/CodeGen/PowerPC/ctrloop-sh.ll | 58 +--
llvm/test/CodeGen/PowerPC/pr59074.ll | 21 +-
...lar-shift-by-byte-multiple-legalization.ll | 48 +--
.../PowerPC/wide-scalar-shift-legalization.ll | 33 +-
...calling-conv-ilp32-ilp32f-ilp32d-common.ll | 72 ++--
llvm/test/CodeGen/RISCV/bitint-fp-conv-200.ll | 312 +++++++--------
...calling-conv-ilp32-ilp32f-ilp32d-common.ll | 40 +-
.../test/CodeGen/RISCV/calling-conv-ilp32e.ll | 136 +++----
llvm/test/CodeGen/RISCV/forced-atomics.ll | 38 +-
llvm/test/CodeGen/RISCV/fp128.ll | 188 ++++-----
llvm/test/CodeGen/RISCV/fpclamptosat.ll | 48 +--
.../test/CodeGen/RISCV/half-convert-strict.ll | 100 ++---
llvm/test/CodeGen/RISCV/half-convert.ll | 200 ++++-----
llvm/test/CodeGen/RISCV/idiv_large.ll | 246 ++++++------
llvm/test/CodeGen/RISCV/llvm.frexp.ll | 36 +-
.../CodeGen/RISCV/musttail-indirect-args.ll | 154 +++----
llvm/test/CodeGen/RISCV/shifts.ll | 88 ++--
llvm/test/CodeGen/RISCV/stack-store-check.ll | 378 +++++++++---------
...lar-shift-by-byte-multiple-legalization.ll | 228 +++++------
.../RISCV/wide-scalar-shift-legalization.ll | 122 +++---
.../Thumb/umulo-128-legalisation-lowering.ll | 196 ++++-----
.../Thumb2/umulo-128-legalisation-lowering.ll | 174 ++++----
.../BoundsChecking/runtimes.ll | 131 +++---
.../Instrumentation/BoundsChecking/simple.ll | 30 +-
.../Instrumentation/DataFlowSanitizer/load.ll | 2 +-
.../DataFlowSanitizer/origin_load.ll | 10 +-
.../test/Instrumentation/Instrumentor/cast.ll | 6 +-
.../Instrumentor/cast_crash.ll | 10 +-
.../Instrumentor/load_store_gpu_ind.ll | 8 +-
.../Instrumentation/Instrumentor/numeric.ll | 42 +-
.../MemorySanitizer/PowerPC32/kernel-ppcle.ll | 12 +-
.../Instrumentation/MemorySanitizer/byval.ll | 14 +-
.../MemorySanitizer/msan_kernel_basic.ll | 14 +-
.../Transforms/AtomicExpand/SPARC/libcalls.ll | 44 +-
.../lifetime-markers-on-inputs-2.ll | 6 +-
.../InferAlignment/irregular-size.ll | 6 +-
llvm/test/Transforms/InstCombine/apint-and.ll | 2 +-
.../InstCombine/load-store-forward.ll | 4 +-
llvm/test/Transforms/InstCombine/or-xor.ll | 2 +-
llvm/test/Transforms/InstCombine/select.ll | 13 +-
llvm/test/Transforms/InstCombine/shift.ll | 16 +-
llvm/test/Transforms/InstCombine/xor.ll | 2 +-
.../LoopIdiom/X86/unordered-atomic-memcpy.ll | 8 +-
.../Transforms/LowerMatrixIntrinsics/unary.ll | 4 +-
llvm/test/Transforms/SCCP/apint-bigint2.ll | 4 +-
llvm/test/Transforms/SCCP/ub-shift.ll | 18 +-
...hreading-max-jump-threading-live-blocks.ll | 8 +-
llvm/test/tools/llubi/loadstore_be.ll | 2 +-
llvm/test/tools/llubi/loadstore_le.ll | 2 +-
.../Frontend/OpenMPIRBuilderTest.cpp | 2 +-
mlir/test/Target/LLVMIR/openmp-llvm.mlir | 2 +-
.../multiple-types-non-power-of-two-2.ll | 2 +-
.../multiple-types-non-power-of-two.ll | 4 +-
108 files changed, 3975 insertions(+), 3215 deletions(-)
diff --git a/clang/test/CodeGen/LoongArch/bitint.c b/clang/test/CodeGen/LoongArch/bitint.c
index 5ed8c575b99d3..cee06259a5dad 100644
--- a/clang/test/CodeGen/LoongArch/bitint.c
+++ b/clang/test/CodeGen/LoongArch/bitint.c
@@ -13,9 +13,9 @@ void pass_BitInt129(_BitInt(129));
// LA32-NEXT: [[ENTRY:.*:]]
// LA32-NEXT: [[L7:%.*]] = alloca i8, align 1
// LA32-NEXT: [[L65:%.*]] = alloca i128, align 16
-// LA32-NEXT: [[L129:%.*]] = alloca [32 x i8], align 16
+// LA32-NEXT: [[L129:%.*]] = alloca i256, align 16
// LA32-NEXT: [[BYVAL_TEMP:%.*]] = alloca i128, align 16
-// LA32-NEXT: [[BYVAL_TEMP3:%.*]] = alloca [32 x i8], align 16
+// LA32-NEXT: [[BYVAL_TEMP3:%.*]] = alloca i256, align 16
// LA32-NEXT: store i8 0, ptr [[L7]], align 1
// LA32-NEXT: store i128 0, ptr [[L65]], align 16
// LA32-NEXT: store i256 0, ptr [[L129]], align 16
@@ -96,7 +96,7 @@ _BitInt(7) return_large_BitInt7(void) { return 0; }
//
_BitInt(65) return_large_BitInt65(void) { return 0; }
// LA32-LABEL: define dso_local void @return_large_BitInt129(
-// LA32-SAME: ptr dead_on_unwind noalias writable sret([32 x i8]) align 16 [[AGG_RESULT:%.*]]) #[[ATTR0]] {
+// LA32-SAME: ptr dead_on_unwind noalias writable sret(i256) align 16 [[AGG_RESULT:%.*]]) #[[ATTR0]] {
// LA32-NEXT: [[ENTRY:.*:]]
// LA32-NEXT: [[RESULT_PTR:%.*]] = alloca ptr, align 4
// LA32-NEXT: store ptr [[AGG_RESULT]], ptr [[RESULT_PTR]], align 4
diff --git a/clang/test/CodeGenHIP/printf_nonhostcall.cpp b/clang/test/CodeGenHIP/printf_nonhostcall.cpp
index 7e8a468a33527..fd16f685ad0af 100644
--- a/clang/test/CodeGenHIP/printf_nonhostcall.cpp
+++ b/clang/test/CodeGenHIP/printf_nonhostcall.cpp
@@ -300,7 +300,7 @@ __device__ _BitInt(128) Int128 = 45637;
// CHECK-NEXT: [[TMP19:%.*]] = zext i44 [[LOADEDV2]] to i64
// CHECK-NEXT: store i64 [[TMP19]], ptr addrspace(1) [[PRINTBUFFNEXTPTR10]], align 8
// CHECK-NEXT: [[PRINTBUFFNEXTPTR11:%.*]] = getelementptr inbounds i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR10]], i32 8
-// CHECK-NEXT: store i128 [[TMP9]], ptr addrspace(1) [[PRINTBUFFNEXTPTR11]], align 8
+// CHECK-NEXT: store i128 [[TMP9]], ptr addrspace(1) [[PRINTBUFFNEXTPTR11]], align 16
// CHECK-NEXT: [[PRINTBUFFNEXTPTR12:%.*]] = getelementptr inbounds i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR11]], i32 16
// CHECK-NEXT: br label [[END_BLOCK]]
//
@@ -359,7 +359,7 @@ __device__ _BitInt(128) Int128 = 45637;
// CHECK_CONSTRAINED-NEXT: [[TMP19:%.*]] = zext i44 [[LOADEDV2]] to i64
// CHECK_CONSTRAINED-NEXT: store i64 [[TMP19]], ptr addrspace(1) [[PRINTBUFFNEXTPTR10]], align 8
// CHECK_CONSTRAINED-NEXT: [[PRINTBUFFNEXTPTR11:%.*]] = getelementptr inbounds i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR10]], i32 8
-// CHECK_CONSTRAINED-NEXT: store i128 [[TMP9]], ptr addrspace(1) [[PRINTBUFFNEXTPTR11]], align 8
+// CHECK_CONSTRAINED-NEXT: store i128 [[TMP9]], ptr addrspace(1) [[PRINTBUFFNEXTPTR11]], align 16
// CHECK_CONSTRAINED-NEXT: [[PRINTBUFFNEXTPTR12:%.*]] = getelementptr inbounds i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR11]], i32 16
// CHECK_CONSTRAINED-NEXT: br label [[END_BLOCK]]
//
diff --git a/llvm/test/Analysis/CostModel/AMDGPU/load-to-trunc.ll b/llvm/test/Analysis/CostModel/AMDGPU/load-to-trunc.ll
index 92412d706d224..6ed945682b9a2 100644
--- a/llvm/test/Analysis/CostModel/AMDGPU/load-to-trunc.ll
+++ b/llvm/test/Analysis/CostModel/AMDGPU/load-to-trunc.ll
@@ -8,7 +8,7 @@
; Check that cost is 1 for unusual load to register sized load.
define i32 @loadUnusualIntegerWithTrunc(ptr %ptr) {
; CHECK-LABEL: 'loadUnusualIntegerWithTrunc'
-; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %out = load i128, ptr %ptr, align 8
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %out = load i128, ptr %ptr, align 16
; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %trunc = trunc i128 %out to i32
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i32 %trunc
;
@@ -19,7 +19,7 @@ define i32 @loadUnusualIntegerWithTrunc(ptr %ptr) {
define i128 @loadUnusualInteger(ptr %ptr) {
; CHECK-LABEL: 'loadUnusualInteger'
-; CHECK-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %out = load i128, ptr %ptr, align 8
+; CHECK-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %out = load i128, ptr %ptr, align 16
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i128 %out
;
%out = load i128, ptr %ptr
diff --git a/llvm/test/Analysis/CostModel/ARM/load-to-trunc.ll b/llvm/test/Analysis/CostModel/ARM/load-to-trunc.ll
index c98601ff45745..70d3dd8f0a042 100644
--- a/llvm/test/Analysis/CostModel/ARM/load-to-trunc.ll
+++ b/llvm/test/Analysis/CostModel/ARM/load-to-trunc.ll
@@ -9,7 +9,7 @@
; Check that cost is 1 for unusual load to register sized load.
define i32 @loadUnusualIntegerWithTrunc(ptr %ptr) {
; CHECK-LABEL: 'loadUnusualIntegerWithTrunc'
-; CHECK-NEXT: Cost Model: Found costs of RThru:4 CodeSize:1 Lat:4 SizeLat:1 for: %out = load i128, ptr %ptr, align 8
+; CHECK-NEXT: Cost Model: Found costs of RThru:4 CodeSize:1 Lat:4 SizeLat:1 for: %out = load i128, ptr %ptr, align 16
; CHECK-NEXT: Cost Model: Found costs of 0 for: %trunc = trunc i128 %out to i32
; CHECK-NEXT: Cost Model: Found costs of 1 for: ret i32 %trunc
;
@@ -20,7 +20,7 @@ define i32 @loadUnusualIntegerWithTrunc(ptr %ptr) {
define i128 @loadUnusualInteger(ptr %ptr) {
; CHECK-LABEL: 'loadUnusualInteger'
-; CHECK-NEXT: Cost Model: Found costs of RThru:4 CodeSize:1 Lat:4 SizeLat:1 for: %out = load i128, ptr %ptr, align 8
+; CHECK-NEXT: Cost Model: Found costs of RThru:4 CodeSize:1 Lat:4 SizeLat:1 for: %out = load i128, ptr %ptr, align 16
; CHECK-NEXT: Cost Model: Found costs of 1 for: ret i128 %out
;
%out = load i128, ptr %ptr
diff --git a/llvm/test/Analysis/LoopAccessAnalysis/depend_diff_types.ll b/llvm/test/Analysis/LoopAccessAnalysis/depend_diff_types.ll
index 84e9d70a16d3b..c2d85f399a912 100644
--- a/llvm/test/Analysis/LoopAccessAnalysis/depend_diff_types.ll
+++ b/llvm/test/Analysis/LoopAccessAnalysis/depend_diff_types.ll
@@ -373,7 +373,7 @@ define void @different_type_sizes_strided_accesses_store_size_exceeds_depdist(pt
; CHECK-NEXT: Dependences:
; CHECK-NEXT: IndirectUnsafe:
; CHECK-NEXT: store i16 0, ptr %gep.iv, align 2 ->
-; CHECK-NEXT: store i128 1, ptr %gep.10.iv, align 4
+; CHECK-NEXT: store i128 1, ptr %gep.10.iv, align 16
; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/function-returns.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/function-returns.ll
index 898e987f4a826..706b6580b1ab0 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/function-returns.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/function-returns.ll
@@ -297,7 +297,7 @@ define i65 @i65_func_void() #0 {
; CHECK-LABEL: name: i65_func_void
; CHECK: bb.1 (%ir-block.0):
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i65) = G_LOAD [[DEF]](p1) :: (load (i65) from `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i65) = G_LOAD [[DEF]](p1) :: (load (i65) from `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: [[ANYEXT:%[0-9]+]]:_(i96) = G_ANYEXT [[LOAD]](i65)
; CHECK-NEXT: [[UV:%[0-9]+]]:_(i32), [[UV1:%[0-9]+]]:_(i32), [[UV2:%[0-9]+]]:_(i32) = G_UNMERGE_VALUES [[ANYEXT]](i96)
; CHECK-NEXT: $vgpr0 = COPY [[UV]](i32)
@@ -312,7 +312,7 @@ define signext i65 @i65_signext_func_void() #0 {
; CHECK-LABEL: name: i65_signext_func_void
; CHECK: bb.1 (%ir-block.0):
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i65) = G_LOAD [[DEF]](p1) :: (load (i65) from `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i65) = G_LOAD [[DEF]](p1) :: (load (i65) from `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: [[SEXT:%[0-9]+]]:_(i96) = G_SEXT [[LOAD]](i65)
; CHECK-NEXT: [[UV:%[0-9]+]]:_(i32), [[UV1:%[0-9]+]]:_(i32), [[UV2:%[0-9]+]]:_(i32) = G_UNMERGE_VALUES [[SEXT]](i96)
; CHECK-NEXT: $vgpr0 = COPY [[UV]](i32)
@@ -327,7 +327,7 @@ define zeroext i65 @i65_zeroext_func_void() #0 {
; CHECK-LABEL: name: i65_zeroext_func_void
; CHECK: bb.1 (%ir-block.0):
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i65) = G_LOAD [[DEF]](p1) :: (load (i65) from `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i65) = G_LOAD [[DEF]](p1) :: (load (i65) from `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: [[ZEXT:%[0-9]+]]:_(i96) = G_ZEXT [[LOAD]](i65)
; CHECK-NEXT: [[UV:%[0-9]+]]:_(i32), [[UV1:%[0-9]+]]:_(i32), [[UV2:%[0-9]+]]:_(i32) = G_UNMERGE_VALUES [[ZEXT]](i96)
; CHECK-NEXT: $vgpr0 = COPY [[UV]](i32)
@@ -1157,7 +1157,7 @@ define i1022 @i1022_func_void() #0 {
; CHECK-LABEL: name: i1022_func_void
; CHECK: bb.1 (%ir-block.0):
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i1022) = G_LOAD [[DEF]](p1) :: (load (i1022) from `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i1022) = G_LOAD [[DEF]](p1) :: (load (i1022) from `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: [[ANYEXT:%[0-9]+]]:_(i1024) = G_ANYEXT [[LOAD]](i1022)
; CHECK-NEXT: [[UV:%[0-9]+]]:_(i32), [[UV1:%[0-9]+]]:_(i32), [[UV2:%[0-9]+]]:_(i32), [[UV3:%[0-9]+]]:_(i32), [[UV4:%[0-9]+]]:_(i32), [[UV5:%[0-9]+]]:_(i32), [[UV6:%[0-9]+]]:_(i32), [[UV7:%[0-9]+]]:_(i32), [[UV8:%[0-9]+]]:_(i32), [[UV9:%[0-9]+]]:_(i32), [[UV10:%[0-9]+]]:_(i32), [[UV11:%[0-9]+]]:_(i32), [[UV12:%[0-9]+]]:_(i32), [[UV13:%[0-9]+]]:_(i32), [[UV14:%[0-9]+]]:_(i32), [[UV15:%[0-9]+]]:_(i32), [[UV16:%[0-9]+]]:_(i32), [[UV17:%[0-9]+]]:_(i32), [[UV18:%[0-9]+]]:_(i32), [[UV19:%[0-9]+]]:_(i32), [[UV20:%[0-9]+]]:_(i32), [[UV21:%[0-9]+]]:_(i32), [[UV22:%[0-9]+]]:_(i32), [[UV23:%[0-9]+]]:_(i32), [[UV24:%[0-9]+]]:_(i32), [[UV25:%[0-9]+]]:_(i32), [[UV26:%[0-9]+]]:_(i32), [[UV27:%[0-9]+]]:_(i32), [[UV28:%[0-9]+]]:_(i32), [[UV29:%[0-9]+]]:_(i32), [[UV30:%[0-9]+]]:_(i32), [[UV31:%[0-9]+]]:_(i32) = G_UNMERGE_VALUES [[ANYEXT]](i1024)
; CHECK-NEXT: $vgpr0 = COPY [[UV]](i32)
@@ -1201,7 +1201,7 @@ define signext i1022 @i1022_signext_func_void() #0 {
; CHECK-LABEL: name: i1022_signext_func_void
; CHECK: bb.1 (%ir-block.0):
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i1022) = G_LOAD [[DEF]](p1) :: (load (i1022) from `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i1022) = G_LOAD [[DEF]](p1) :: (load (i1022) from `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: [[SEXT:%[0-9]+]]:_(i1024) = G_SEXT [[LOAD]](i1022)
; CHECK-NEXT: [[UV:%[0-9]+]]:_(i32), [[UV1:%[0-9]+]]:_(i32), [[UV2:%[0-9]+]]:_(i32), [[UV3:%[0-9]+]]:_(i32), [[UV4:%[0-9]+]]:_(i32), [[UV5:%[0-9]+]]:_(i32), [[UV6:%[0-9]+]]:_(i32), [[UV7:%[0-9]+]]:_(i32), [[UV8:%[0-9]+]]:_(i32), [[UV9:%[0-9]+]]:_(i32), [[UV10:%[0-9]+]]:_(i32), [[UV11:%[0-9]+]]:_(i32), [[UV12:%[0-9]+]]:_(i32), [[UV13:%[0-9]+]]:_(i32), [[UV14:%[0-9]+]]:_(i32), [[UV15:%[0-9]+]]:_(i32), [[UV16:%[0-9]+]]:_(i32), [[UV17:%[0-9]+]]:_(i32), [[UV18:%[0-9]+]]:_(i32), [[UV19:%[0-9]+]]:_(i32), [[UV20:%[0-9]+]]:_(i32), [[UV21:%[0-9]+]]:_(i32), [[UV22:%[0-9]+]]:_(i32), [[UV23:%[0-9]+]]:_(i32), [[UV24:%[0-9]+]]:_(i32), [[UV25:%[0-9]+]]:_(i32), [[UV26:%[0-9]+]]:_(i32), [[UV27:%[0-9]+]]:_(i32), [[UV28:%[0-9]+]]:_(i32), [[UV29:%[0-9]+]]:_(i32), [[UV30:%[0-9]+]]:_(i32), [[UV31:%[0-9]+]]:_(i32) = G_UNMERGE_VALUES [[SEXT]](i1024)
; CHECK-NEXT: $vgpr0 = COPY [[UV]](i32)
@@ -1245,7 +1245,7 @@ define zeroext i1022 @i1022_zeroext_func_void() #0 {
; CHECK-LABEL: name: i1022_zeroext_func_void
; CHECK: bb.1 (%ir-block.0):
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i1022) = G_LOAD [[DEF]](p1) :: (load (i1022) from `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: [[LOAD:%[0-9]+]]:_(i1022) = G_LOAD [[DEF]](p1) :: (load (i1022) from `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: [[ZEXT:%[0-9]+]]:_(i1024) = G_ZEXT [[LOAD]](i1022)
; CHECK-NEXT: [[UV:%[0-9]+]]:_(i32), [[UV1:%[0-9]+]]:_(i32), [[UV2:%[0-9]+]]:_(i32), [[UV3:%[0-9]+]]:_(i32), [[UV4:%[0-9]+]]:_(i32), [[UV5:%[0-9]+]]:_(i32), [[UV6:%[0-9]+]]:_(i32), [[UV7:%[0-9]+]]:_(i32), [[UV8:%[0-9]+]]:_(i32), [[UV9:%[0-9]+]]:_(i32), [[UV10:%[0-9]+]]:_(i32), [[UV11:%[0-9]+]]:_(i32), [[UV12:%[0-9]+]]:_(i32), [[UV13:%[0-9]+]]:_(i32), [[UV14:%[0-9]+]]:_(i32), [[UV15:%[0-9]+]]:_(i32), [[UV16:%[0-9]+]]:_(i32), [[UV17:%[0-9]+]]:_(i32), [[UV18:%[0-9]+]]:_(i32), [[UV19:%[0-9]+]]:_(i32), [[UV20:%[0-9]+]]:_(i32), [[UV21:%[0-9]+]]:_(i32), [[UV22:%[0-9]+]]:_(i32), [[UV23:%[0-9]+]]:_(i32), [[UV24:%[0-9]+]]:_(i32), [[UV25:%[0-9]+]]:_(i32), [[UV26:%[0-9]+]]:_(i32), [[UV27:%[0-9]+]]:_(i32), [[UV28:%[0-9]+]]:_(i32), [[UV29:%[0-9]+]]:_(i32), [[UV30:%[0-9]+]]:_(i32), [[UV31:%[0-9]+]]:_(i32) = G_UNMERGE_VALUES [[ZEXT]](i1024)
; CHECK-NEXT: $vgpr0 = COPY [[UV]](i32)
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/irtranslator-function-args.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/irtranslator-function-args.ll
index 25f952df63f29..c3f46dd877115 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/irtranslator-function-args.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/irtranslator-function-args.ll
@@ -412,7 +412,7 @@ define void @void_func_i95(i95 %arg0) #0 {
; CHECK-NEXT: [[MV:%[0-9]+]]:_(i96) = G_MERGE_VALUES [[COPY]](i32), [[COPY1]](i32), [[COPY2]](i32)
; CHECK-NEXT: [[TRUNC:%[0-9]+]]:_(i95) = G_TRUNC [[MV]](i96)
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: G_STORE [[TRUNC]](i95), [[DEF]](p1) :: (store (i95) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: G_STORE [[TRUNC]](i95), [[DEF]](p1) :: (store (i95) into `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: SI_RETURN
store i95 %arg0, ptr addrspace(1) poison
ret void
@@ -432,7 +432,7 @@ define void @void_func_i95_zeroext(i95 zeroext %arg0) #0 {
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
; CHECK-NEXT: [[ZEXT:%[0-9]+]]:_(i96) = G_ZEXT [[TRUNC]](i95)
; CHECK-NEXT: [[ADD:%[0-9]+]]:_(i96) = G_ADD [[ZEXT]], [[C]]
- ; CHECK-NEXT: G_STORE [[ADD]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: G_STORE [[ADD]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: SI_RETURN
%ext = zext i95 %arg0 to i96
%add = add i96 %ext, 12
@@ -454,7 +454,7 @@ define void @void_func_i95_signext(i95 signext %arg0) #0 {
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
; CHECK-NEXT: [[SEXT:%[0-9]+]]:_(i96) = G_SEXT [[TRUNC]](i95)
; CHECK-NEXT: [[ADD:%[0-9]+]]:_(i96) = G_ADD [[SEXT]], [[C]]
- ; CHECK-NEXT: G_STORE [[ADD]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: G_STORE [[ADD]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: SI_RETURN
%ext = sext i95 %arg0 to i96
%add = add i96 %ext, 12
@@ -472,7 +472,7 @@ define void @void_func_i96(i96 %arg0) #0 {
; CHECK-NEXT: [[COPY2:%[0-9]+]]:_(i32) = COPY $vgpr2
; CHECK-NEXT: [[MV:%[0-9]+]]:_(i96) = G_MERGE_VALUES [[COPY]](i32), [[COPY1]](i32), [[COPY2]](i32)
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: G_STORE [[MV]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: G_STORE [[MV]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: SI_RETURN
store i96 %arg0, ptr addrspace(1) poison
ret void
@@ -2875,7 +2875,7 @@ define void @void_func_i96_inreg(i96 inreg %arg0) #0 {
; CHECK-NEXT: [[COPY2:%[0-9]+]]:_(i32) = COPY $sgpr18
; CHECK-NEXT: [[MV:%[0-9]+]]:_(i96) = G_MERGE_VALUES [[COPY]](i32), [[COPY1]](i32), [[COPY2]](i32)
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: G_STORE [[MV]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: G_STORE [[MV]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 16, addrspace 1)
; CHECK-NEXT: SI_RETURN
store i96 %arg0, ptr addrspace(1) poison
ret void
@@ -2892,7 +2892,7 @@ define void @void_func_i128_inreg(i128 inreg %arg0) #0 {
; CHECK-NEXT: [[COPY3:%[0-9]+]]:_(i32) = COPY $sgpr19
; CHECK-NEXT: [[MV:%[0-9]+]]:_(i128) = G_MERGE_VALUES [[COPY]](i32), [[COPY1]](i32), [[COPY2]](i32), [[COPY3]](i32)
; CHECK-NEXT: [[DEF:%[0-9]+]]:_(p1) = G_IMPLICIT_DEF
- ; CHECK-NEXT: G_STORE [[MV]](i128), [[DEF]](p1) :: (store (i128) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; CHECK-NEXT: G_STORE [[MV]](i128), [[DEF]](p1) :: (store (i128) into `ptr addrspace(1) poison`, addrspace 1)
; CHECK-NEXT: SI_RETURN
store i128 %arg0, ptr addrspace(1) poison
ret void
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.s.buffer.load.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.s.buffer.load.ll
index 8e985b16d322f..0f466f4272726 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.s.buffer.load.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.s.buffer.load.ll
@@ -659,10 +659,10 @@ define amdgpu_ps void @s_buffer_load_i96_vgpr_offset(<4 x i32> inreg %rsrc, i32
; GFX7-NEXT: [[DEF:%[0-9]+]]:sgpr(p1) = G_IMPLICIT_DEF
; GFX7-NEXT: [[C:%[0-9]+]]:sgpr(i32) = G_CONSTANT i32 0
; GFX7-NEXT: [[C1:%[0-9]+]]:vgpr(i32) = G_CONSTANT i32 0
- ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(s128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s96), align 8)
+ ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(s128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s96), align 16)
; GFX7-NEXT: [[COPY5:%[0-9]+]]:vgpr(s128) = COPY [[AMDGPU_BUFFER_LOAD]](s128)
; GFX7-NEXT: [[TRUNC:%[0-9]+]]:vgpr(i96) = G_TRUNC [[COPY5]](s128)
- ; GFX7-NEXT: G_STORE [[TRUNC]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; GFX7-NEXT: G_STORE [[TRUNC]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 16, addrspace 1)
; GFX7-NEXT: S_ENDPGM 0
;
; GFX1200_1250-LABEL: name: s_buffer_load_i96_vgpr_offset
@@ -678,9 +678,9 @@ define amdgpu_ps void @s_buffer_load_i96_vgpr_offset(<4 x i32> inreg %rsrc, i32
; GFX1200_1250-NEXT: [[DEF:%[0-9]+]]:sgpr(p1) = G_IMPLICIT_DEF
; GFX1200_1250-NEXT: [[C:%[0-9]+]]:sgpr(i32) = G_CONSTANT i32 0
; GFX1200_1250-NEXT: [[C1:%[0-9]+]]:vgpr(i32) = G_CONSTANT i32 0
- ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i96) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s96), align 8)
+ ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i96) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s96), align 16)
; GFX1200_1250-NEXT: [[COPY5:%[0-9]+]]:vgpr(i96) = COPY [[AMDGPU_BUFFER_LOAD]](i96)
- ; GFX1200_1250-NEXT: G_STORE [[COPY5]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; GFX1200_1250-NEXT: G_STORE [[COPY5]](i96), [[DEF]](p1) :: (store (i96) into `ptr addrspace(1) poison`, align 16, addrspace 1)
; GFX1200_1250-NEXT: S_ENDPGM 0
%val = call i96 @llvm.amdgcn.s.buffer.load.i96(<4 x i32> %rsrc, i32 %soffset, i32 0)
store i96 %val, ptr addrspace(1) poison
@@ -702,14 +702,14 @@ define amdgpu_ps void @s_buffer_load_i256_vgpr_offset(<4 x i32> inreg %rsrc, i32
; GFX7-NEXT: [[DEF:%[0-9]+]]:sgpr(p1) = G_IMPLICIT_DEF
; GFX7-NEXT: [[C:%[0-9]+]]:sgpr(i32) = G_CONSTANT i32 0
; GFX7-NEXT: [[C1:%[0-9]+]]:vgpr(i32) = G_CONSTANT i32 0
- ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s128), align 8)
- ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD1:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 16, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 16, align 8)
+ ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s128))
+ ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD1:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 16, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 16)
; GFX7-NEXT: [[MV:%[0-9]+]]:vgpr(i256) = G_MERGE_VALUES [[AMDGPU_BUFFER_LOAD]](i128), [[AMDGPU_BUFFER_LOAD1]](i128)
; GFX7-NEXT: [[UV:%[0-9]+]]:vgpr(s128), [[UV1:%[0-9]+]]:vgpr(s128) = G_UNMERGE_VALUES [[MV]](i256)
- ; GFX7-NEXT: G_STORE [[UV]](s128), [[DEF]](p1) :: (store (s128) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; GFX7-NEXT: G_STORE [[UV]](s128), [[DEF]](p1) :: (store (s128) into `ptr addrspace(1) poison`, addrspace 1)
; GFX7-NEXT: [[C2:%[0-9]+]]:sgpr(i64) = G_CONSTANT i64 16
; GFX7-NEXT: [[PTR_ADD:%[0-9]+]]:sgpr(p1) = nuw inbounds G_PTR_ADD [[DEF]], [[C2]](i64)
- ; GFX7-NEXT: G_STORE [[UV1]](s128), [[PTR_ADD]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 16, align 8, addrspace 1)
+ ; GFX7-NEXT: G_STORE [[UV1]](s128), [[PTR_ADD]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 16, addrspace 1)
; GFX7-NEXT: S_ENDPGM 0
;
; GFX1200_1250-LABEL: name: s_buffer_load_i256_vgpr_offset
@@ -725,14 +725,14 @@ define amdgpu_ps void @s_buffer_load_i256_vgpr_offset(<4 x i32> inreg %rsrc, i32
; GFX1200_1250-NEXT: [[DEF:%[0-9]+]]:sgpr(p1) = G_IMPLICIT_DEF
; GFX1200_1250-NEXT: [[C:%[0-9]+]]:sgpr(i32) = G_CONSTANT i32 0
; GFX1200_1250-NEXT: [[C1:%[0-9]+]]:vgpr(i32) = G_CONSTANT i32 0
- ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s128), align 8)
- ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD1:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 16, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 16, align 8)
+ ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s128))
+ ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD1:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 16, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 16)
; GFX1200_1250-NEXT: [[MV:%[0-9]+]]:vgpr(i256) = G_MERGE_VALUES [[AMDGPU_BUFFER_LOAD]](i128), [[AMDGPU_BUFFER_LOAD1]](i128)
; GFX1200_1250-NEXT: [[UV:%[0-9]+]]:vgpr(s128), [[UV1:%[0-9]+]]:vgpr(s128) = G_UNMERGE_VALUES [[MV]](i256)
- ; GFX1200_1250-NEXT: G_STORE [[UV]](s128), [[DEF]](p1) :: (store (s128) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; GFX1200_1250-NEXT: G_STORE [[UV]](s128), [[DEF]](p1) :: (store (s128) into `ptr addrspace(1) poison`, addrspace 1)
; GFX1200_1250-NEXT: [[C2:%[0-9]+]]:sgpr(i64) = G_CONSTANT i64 16
; GFX1200_1250-NEXT: [[PTR_ADD:%[0-9]+]]:sgpr(p1) = nuw inbounds G_PTR_ADD [[DEF]], [[C2]](i64)
- ; GFX1200_1250-NEXT: G_STORE [[UV1]](s128), [[PTR_ADD]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 16, align 8, addrspace 1)
+ ; GFX1200_1250-NEXT: G_STORE [[UV1]](s128), [[PTR_ADD]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 16, addrspace 1)
; GFX1200_1250-NEXT: S_ENDPGM 0
%val = call i256 @llvm.amdgcn.s.buffer.load.i256(<4 x i32> %rsrc, i32 %soffset, i32 0)
store i256 %val, ptr addrspace(1) poison
@@ -754,22 +754,22 @@ define amdgpu_ps void @s_buffer_load_i512_vgpr_offset(<4 x i32> inreg %rsrc, i32
; GFX7-NEXT: [[DEF:%[0-9]+]]:sgpr(p1) = G_IMPLICIT_DEF
; GFX7-NEXT: [[C:%[0-9]+]]:sgpr(i32) = G_CONSTANT i32 0
; GFX7-NEXT: [[C1:%[0-9]+]]:vgpr(i32) = G_CONSTANT i32 0
- ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s128), align 8)
- ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD1:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 16, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 16, align 8)
- ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD2:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 32, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 32, align 8)
- ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD3:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 48, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 48, align 8)
+ ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s128))
+ ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD1:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 16, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 16)
+ ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD2:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 32, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 32)
+ ; GFX7-NEXT: [[AMDGPU_BUFFER_LOAD3:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 48, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 48)
; GFX7-NEXT: [[MV:%[0-9]+]]:vgpr(i512) = G_MERGE_VALUES [[AMDGPU_BUFFER_LOAD]](i128), [[AMDGPU_BUFFER_LOAD1]](i128), [[AMDGPU_BUFFER_LOAD2]](i128), [[AMDGPU_BUFFER_LOAD3]](i128)
; GFX7-NEXT: [[UV:%[0-9]+]]:vgpr(s128), [[UV1:%[0-9]+]]:vgpr(s128), [[UV2:%[0-9]+]]:vgpr(s128), [[UV3:%[0-9]+]]:vgpr(s128) = G_UNMERGE_VALUES [[MV]](i512)
- ; GFX7-NEXT: G_STORE [[UV]](s128), [[DEF]](p1) :: (store (s128) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; GFX7-NEXT: G_STORE [[UV]](s128), [[DEF]](p1) :: (store (s128) into `ptr addrspace(1) poison`, addrspace 1)
; GFX7-NEXT: [[C2:%[0-9]+]]:sgpr(i64) = G_CONSTANT i64 16
; GFX7-NEXT: [[PTR_ADD:%[0-9]+]]:sgpr(p1) = nuw inbounds G_PTR_ADD [[DEF]], [[C2]](i64)
- ; GFX7-NEXT: G_STORE [[UV1]](s128), [[PTR_ADD]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 16, align 8, addrspace 1)
+ ; GFX7-NEXT: G_STORE [[UV1]](s128), [[PTR_ADD]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 16, addrspace 1)
; GFX7-NEXT: [[C3:%[0-9]+]]:sgpr(i64) = G_CONSTANT i64 32
; GFX7-NEXT: [[PTR_ADD1:%[0-9]+]]:sgpr(p1) = nuw inbounds G_PTR_ADD [[DEF]], [[C3]](i64)
- ; GFX7-NEXT: G_STORE [[UV2]](s128), [[PTR_ADD1]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 32, align 8, addrspace 1)
+ ; GFX7-NEXT: G_STORE [[UV2]](s128), [[PTR_ADD1]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 32, addrspace 1)
; GFX7-NEXT: [[C4:%[0-9]+]]:sgpr(i64) = G_CONSTANT i64 48
; GFX7-NEXT: [[PTR_ADD2:%[0-9]+]]:sgpr(p1) = nuw inbounds G_PTR_ADD [[DEF]], [[C4]](i64)
- ; GFX7-NEXT: G_STORE [[UV3]](s128), [[PTR_ADD2]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 48, align 8, addrspace 1)
+ ; GFX7-NEXT: G_STORE [[UV3]](s128), [[PTR_ADD2]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 48, addrspace 1)
; GFX7-NEXT: S_ENDPGM 0
;
; GFX1200_1250-LABEL: name: s_buffer_load_i512_vgpr_offset
@@ -785,22 +785,22 @@ define amdgpu_ps void @s_buffer_load_i512_vgpr_offset(<4 x i32> inreg %rsrc, i32
; GFX1200_1250-NEXT: [[DEF:%[0-9]+]]:sgpr(p1) = G_IMPLICIT_DEF
; GFX1200_1250-NEXT: [[C:%[0-9]+]]:sgpr(i32) = G_CONSTANT i32 0
; GFX1200_1250-NEXT: [[C1:%[0-9]+]]:vgpr(i32) = G_CONSTANT i32 0
- ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s128), align 8)
- ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD1:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 16, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 16, align 8)
- ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD2:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 32, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 32, align 8)
- ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD3:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 48, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 48, align 8)
+ ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 0, 0, 0 :: (dereferenceable invariant load (s128))
+ ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD1:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 16, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 16)
+ ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD2:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 32, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 32)
+ ; GFX1200_1250-NEXT: [[AMDGPU_BUFFER_LOAD3:%[0-9]+]]:vgpr(i128) = G_AMDGPU_BUFFER_LOAD [[BUILD_VECTOR]](<4 x i32>), [[C1]](i32), [[COPY4]], [[C]], 48, 0, 0 :: (dereferenceable invariant load (s128) from unknown-address + 48)
; GFX1200_1250-NEXT: [[MV:%[0-9]+]]:vgpr(i512) = G_MERGE_VALUES [[AMDGPU_BUFFER_LOAD]](i128), [[AMDGPU_BUFFER_LOAD1]](i128), [[AMDGPU_BUFFER_LOAD2]](i128), [[AMDGPU_BUFFER_LOAD3]](i128)
; GFX1200_1250-NEXT: [[UV:%[0-9]+]]:vgpr(s128), [[UV1:%[0-9]+]]:vgpr(s128), [[UV2:%[0-9]+]]:vgpr(s128), [[UV3:%[0-9]+]]:vgpr(s128) = G_UNMERGE_VALUES [[MV]](i512)
- ; GFX1200_1250-NEXT: G_STORE [[UV]](s128), [[DEF]](p1) :: (store (s128) into `ptr addrspace(1) poison`, align 8, addrspace 1)
+ ; GFX1200_1250-NEXT: G_STORE [[UV]](s128), [[DEF]](p1) :: (store (s128) into `ptr addrspace(1) poison`, addrspace 1)
; GFX1200_1250-NEXT: [[C2:%[0-9]+]]:sgpr(i64) = G_CONSTANT i64 16
; GFX1200_1250-NEXT: [[PTR_ADD:%[0-9]+]]:sgpr(p1) = nuw inbounds G_PTR_ADD [[DEF]], [[C2]](i64)
- ; GFX1200_1250-NEXT: G_STORE [[UV1]](s128), [[PTR_ADD]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 16, align 8, addrspace 1)
+ ; GFX1200_1250-NEXT: G_STORE [[UV1]](s128), [[PTR_ADD]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 16, addrspace 1)
; GFX1200_1250-NEXT: [[C3:%[0-9]+]]:sgpr(i64) = G_CONSTANT i64 32
; GFX1200_1250-NEXT: [[PTR_ADD1:%[0-9]+]]:sgpr(p1) = nuw inbounds G_PTR_ADD [[DEF]], [[C3]](i64)
- ; GFX1200_1250-NEXT: G_STORE [[UV2]](s128), [[PTR_ADD1]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 32, align 8, addrspace 1)
+ ; GFX1200_1250-NEXT: G_STORE [[UV2]](s128), [[PTR_ADD1]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 32, addrspace 1)
; GFX1200_1250-NEXT: [[C4:%[0-9]+]]:sgpr(i64) = G_CONSTANT i64 48
; GFX1200_1250-NEXT: [[PTR_ADD2:%[0-9]+]]:sgpr(p1) = nuw inbounds G_PTR_ADD [[DEF]], [[C4]](i64)
- ; GFX1200_1250-NEXT: G_STORE [[UV3]](s128), [[PTR_ADD2]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 48, align 8, addrspace 1)
+ ; GFX1200_1250-NEXT: G_STORE [[UV3]](s128), [[PTR_ADD2]](p1) :: (store (s128) into `ptr addrspace(1) poison` + 48, addrspace 1)
; GFX1200_1250-NEXT: S_ENDPGM 0
%val = call i512 @llvm.amdgcn.s.buffer.load.i512(<4 x i32> %rsrc, i32 %soffset, i32 0)
store i512 %val, ptr addrspace(1) poison
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/trunc-brc.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/trunc-brc.ll
index 116719a6e3e7b..1fe8390ea639f 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/trunc-brc.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/trunc-brc.ll
@@ -57,12 +57,13 @@ define amdgpu_ps void @s_trunc_i96_to_i64(ptr addrspace(1) inreg %src, ptr addrs
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:sreg_32 = COPY $sgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:sreg_32 = COPY $sgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:sreg_64_xexec_xnull = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: [[S_LOAD_DWORDX2_IMM:%[0-9]+]]:sreg_64_xexec = S_LOAD_DWORDX2_IMM [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<2 x s32>) from %ir.src, addrspace 1)
- ; GFX-950-NEXT: [[S_LOAD_DWORD_IMM:%[0-9]+]]:sreg_32_xm0_xexec = S_LOAD_DWORD_IMM [[REG_SEQUENCE]], 8, 0 :: ("amdgpu-noclobber" load (s32) from %ir.src + 8, align 8, addrspace 1)
- ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX2_IMM]].sub0
- ; GFX-950-NEXT: [[COPY5:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX2_IMM]].sub1
- ; GFX-950-NEXT: [[REG_SEQUENCE2:%[0-9]+]]:sgpr_96 = REG_SEQUENCE [[COPY4]], %subreg.sub0, [[COPY5]], %subreg.sub1, [[S_LOAD_DWORD_IMM]], %subreg.sub2
- ; GFX-950-NEXT: [[COPY6:%[0-9]+]]:sreg_64 = COPY [[REG_SEQUENCE2]].sub0_sub1, debug-location !4
+ ; GFX-950-NEXT: [[S_LOAD_DWORDX4_IMM:%[0-9]+]]:sgpr_128 = S_LOAD_DWORDX4_IMM [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<4 x s32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub0
+ ; GFX-950-NEXT: [[COPY5:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub1
+ ; GFX-950-NEXT: [[COPY6:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub2
+ ; GFX-950-NEXT: [[COPY7:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub3
+ ; GFX-950-NEXT: [[REG_SEQUENCE2:%[0-9]+]]:sgpr_96 = REG_SEQUENCE [[COPY4]], %subreg.sub0, [[COPY5]], %subreg.sub1, [[COPY6]], %subreg.sub2
+ ; GFX-950-NEXT: [[COPY8:%[0-9]+]]:sreg_64 = COPY [[REG_SEQUENCE2]].sub0_sub1, debug-location !4
%val = load i96, ptr addrspace(1) %src
%trunc = trunc i96 %val to i64, !dbg !4
store i64 %trunc, ptr addrspace(1) %dst
@@ -80,7 +81,7 @@ define void @v_trunc_i96_to_i64(ptr addrspace(1) %src, ptr addrspace(1) %dst) {
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:vgpr_32 = COPY $vgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:vreg_64_align2 = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX3_:%[0-9]+]]:vreg_96_align2 = GLOBAL_LOAD_DWORDX3 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<3 x i32>) from %ir.src, align 8, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX3_:%[0-9]+]]:vreg_96_align2 = GLOBAL_LOAD_DWORDX3 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<3 x i32>) from %ir.src, align 16, addrspace 1)
; GFX-950-NEXT: [[COPY4:%[0-9]+]]:vreg_64_align2 = COPY [[GLOBAL_LOAD_DWORDX3_]].sub0_sub1, debug-location !4
%val = load i96, ptr addrspace(1) %src
%trunc = trunc i96 %val to i64, !dbg !4
@@ -101,8 +102,8 @@ define amdgpu_ps void @s_trunc_i128_to_i96(ptr addrspace(1) inreg %src, ptr addr
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:sreg_32 = COPY $sgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:sreg_32 = COPY $sgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:sreg_64_xexec_xnull = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: early-clobber %9:sgpr_128 = S_LOAD_DWORDX4_IMM_ec [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<4 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sgpr_96 = COPY %9.sub0_sub1_sub2, debug-location !4
+ ; GFX-950-NEXT: [[S_LOAD_DWORDX4_IMM:%[0-9]+]]:sgpr_128 = S_LOAD_DWORDX4_IMM [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<4 x i32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sgpr_96 = COPY [[S_LOAD_DWORDX4_IMM]].sub0_sub1_sub2, debug-location !4
%val = load i128, ptr addrspace(1) %src
%trunc = trunc i128 %val to i96, !dbg !4
store i96 %trunc, ptr addrspace(1) %dst
@@ -120,7 +121,7 @@ define void @v_trunc_i128_to_i96(ptr addrspace(1) %src, ptr addrspace(1) %dst) {
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:vgpr_32 = COPY $vgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:vreg_64_align2 = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, align 8, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, addrspace 1)
; GFX-950-NEXT: [[COPY4:%[0-9]+]]:vreg_96_align2 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub0_sub1_sub2, debug-location !4
%val = load i128, ptr addrspace(1) %src
%trunc = trunc i128 %val to i96, !dbg !4
@@ -141,12 +142,12 @@ define amdgpu_ps void @s_trunc_i160_to_i128(ptr addrspace(1) inreg %src, ptr add
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:sreg_32 = COPY $sgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:sreg_32 = COPY $sgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:sreg_64_xexec_xnull = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: early-clobber %10:sgpr_128 = S_LOAD_DWORDX4_IMM_ec [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<4 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: [[S_LOAD_DWORD_IMM:%[0-9]+]]:sreg_32_xm0_xexec = S_LOAD_DWORD_IMM [[REG_SEQUENCE]], 16, 0 :: ("amdgpu-noclobber" load (i32) from %ir.src + 16, align 8, addrspace 1)
- ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY %10.sub0
- ; GFX-950-NEXT: [[COPY5:%[0-9]+]]:sreg_32 = COPY %10.sub1
- ; GFX-950-NEXT: [[COPY6:%[0-9]+]]:sreg_32 = COPY %10.sub2
- ; GFX-950-NEXT: [[COPY7:%[0-9]+]]:sreg_32 = COPY %10.sub3
+ ; GFX-950-NEXT: [[S_LOAD_DWORDX4_IMM:%[0-9]+]]:sgpr_128 = S_LOAD_DWORDX4_IMM [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<4 x i32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[S_LOAD_DWORD_IMM:%[0-9]+]]:sreg_32_xm0_xexec = S_LOAD_DWORD_IMM [[REG_SEQUENCE]], 16, 0 :: ("amdgpu-noclobber" load (i32) from %ir.src + 16, align 16, addrspace 1)
+ ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub0
+ ; GFX-950-NEXT: [[COPY5:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub1
+ ; GFX-950-NEXT: [[COPY6:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub2
+ ; GFX-950-NEXT: [[COPY7:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub3
; GFX-950-NEXT: [[REG_SEQUENCE2:%[0-9]+]]:sgpr_160 = REG_SEQUENCE [[COPY4]], %subreg.sub0, [[COPY5]], %subreg.sub1, [[COPY6]], %subreg.sub2, [[COPY7]], %subreg.sub3, [[S_LOAD_DWORD_IMM]], %subreg.sub4
; GFX-950-NEXT: [[COPY8:%[0-9]+]]:sgpr_128 = COPY [[REG_SEQUENCE2]].sub0_sub1_sub2_sub3, debug-location !4
%val = load i160, ptr addrspace(1) %src
@@ -166,8 +167,8 @@ define void @v_trunc_i160_to_i128(ptr addrspace(1) %src, ptr addrspace(1) %dst)
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:vgpr_32 = COPY $vgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:vreg_64_align2 = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORD:%[0-9]+]]:vgpr_32 = GLOBAL_LOAD_DWORD [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (i32) from %ir.src + 16, align 8, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORD:%[0-9]+]]:vgpr_32 = GLOBAL_LOAD_DWORD [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (i32) from %ir.src + 16, align 16, addrspace 1)
; GFX-950-NEXT: [[COPY4:%[0-9]+]]:vgpr_32 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub0
; GFX-950-NEXT: [[COPY5:%[0-9]+]]:vgpr_32 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub1
; GFX-950-NEXT: [[COPY6:%[0-9]+]]:vgpr_32 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub2
@@ -193,12 +194,12 @@ define amdgpu_ps void @s_trunc_i192_to_i160(ptr addrspace(1) inreg %src, ptr add
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:sreg_32 = COPY $sgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:sreg_32 = COPY $sgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:sreg_64_xexec_xnull = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: early-clobber %18:sgpr_128 = S_LOAD_DWORDX4_IMM_ec [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<4 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: [[S_LOAD_DWORDX2_IMM:%[0-9]+]]:sreg_64_xexec = S_LOAD_DWORDX2_IMM [[REG_SEQUENCE]], 16, 0 :: ("amdgpu-noclobber" load (<2 x i32>) from %ir.src + 16, addrspace 1)
- ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY %18.sub0
- ; GFX-950-NEXT: [[COPY5:%[0-9]+]]:sreg_32 = COPY %18.sub1
- ; GFX-950-NEXT: [[COPY6:%[0-9]+]]:sreg_32 = COPY %18.sub2
- ; GFX-950-NEXT: [[COPY7:%[0-9]+]]:sreg_32 = COPY %18.sub3
+ ; GFX-950-NEXT: [[S_LOAD_DWORDX4_IMM:%[0-9]+]]:sgpr_128 = S_LOAD_DWORDX4_IMM [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<4 x i32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[S_LOAD_DWORDX2_IMM:%[0-9]+]]:sreg_64_xexec = S_LOAD_DWORDX2_IMM [[REG_SEQUENCE]], 16, 0 :: ("amdgpu-noclobber" load (<2 x i32>) from %ir.src + 16, align 16, addrspace 1)
+ ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub0
+ ; GFX-950-NEXT: [[COPY5:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub1
+ ; GFX-950-NEXT: [[COPY6:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub2
+ ; GFX-950-NEXT: [[COPY7:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub3
; GFX-950-NEXT: [[COPY8:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX2_IMM]].sub0
; GFX-950-NEXT: [[COPY9:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX2_IMM]].sub1
; GFX-950-NEXT: [[REG_SEQUENCE2:%[0-9]+]]:sgpr_192 = REG_SEQUENCE [[COPY4]], %subreg.sub0, [[COPY5]], %subreg.sub1, [[COPY6]], %subreg.sub2, [[COPY7]], %subreg.sub3, [[COPY8]], %subreg.sub4, [[COPY9]], %subreg.sub5
@@ -220,8 +221,8 @@ define void @v_trunc_i192_to_i160(ptr addrspace(1) %src, ptr addrspace(1) %dst)
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:vgpr_32 = COPY $vgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:vreg_64_align2 = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX2_:%[0-9]+]]:vreg_64_align2 = GLOBAL_LOAD_DWORDX2 [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (<2 x i32>) from %ir.src + 16, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX2_:%[0-9]+]]:vreg_64_align2 = GLOBAL_LOAD_DWORDX2 [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (<2 x i32>) from %ir.src + 16, align 16, addrspace 1)
; GFX-950-NEXT: [[COPY4:%[0-9]+]]:vgpr_32 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub0
; GFX-950-NEXT: [[COPY5:%[0-9]+]]:vgpr_32 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub1
; GFX-950-NEXT: [[COPY6:%[0-9]+]]:vgpr_32 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub2
@@ -249,17 +250,18 @@ define amdgpu_ps void @s_trunc_i224_to_i192(ptr addrspace(1) inreg %src, ptr add
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:sreg_32 = COPY $sgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:sreg_32 = COPY $sgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:sreg_64_xexec_xnull = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: early-clobber %16:sgpr_128 = S_LOAD_DWORDX4_IMM_ec [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<4 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: [[S_LOAD_DWORDX2_IMM:%[0-9]+]]:sreg_64_xexec = S_LOAD_DWORDX2_IMM [[REG_SEQUENCE]], 16, 0 :: ("amdgpu-noclobber" load (<2 x s32>) from %ir.src + 16, addrspace 1)
- ; GFX-950-NEXT: [[S_LOAD_DWORD_IMM:%[0-9]+]]:sreg_32_xm0_xexec = S_LOAD_DWORD_IMM [[REG_SEQUENCE]], 24, 0 :: ("amdgpu-noclobber" load (s32) from %ir.src + 24, align 8, addrspace 1)
- ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX2_IMM]].sub0
- ; GFX-950-NEXT: [[COPY5:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX2_IMM]].sub1
- ; GFX-950-NEXT: [[COPY6:%[0-9]+]]:sreg_32 = COPY %16.sub0
- ; GFX-950-NEXT: [[COPY7:%[0-9]+]]:sreg_32 = COPY %16.sub1
- ; GFX-950-NEXT: [[COPY8:%[0-9]+]]:sreg_32 = COPY %16.sub2
- ; GFX-950-NEXT: [[COPY9:%[0-9]+]]:sreg_32 = COPY %16.sub3
- ; GFX-950-NEXT: [[REG_SEQUENCE2:%[0-9]+]]:sgpr_224 = REG_SEQUENCE [[COPY6]], %subreg.sub0, [[COPY7]], %subreg.sub1, [[COPY8]], %subreg.sub2, [[COPY9]], %subreg.sub3, [[COPY4]], %subreg.sub4, [[COPY5]], %subreg.sub5, [[S_LOAD_DWORD_IMM]], %subreg.sub6
- ; GFX-950-NEXT: [[COPY10:%[0-9]+]]:sgpr_192 = COPY [[REG_SEQUENCE2]].sub0_sub1_sub2_sub3_sub4_sub5, debug-location !4
+ ; GFX-950-NEXT: [[S_LOAD_DWORDX4_IMM:%[0-9]+]]:sgpr_128 = S_LOAD_DWORDX4_IMM [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<4 x i32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[S_LOAD_DWORDX4_IMM1:%[0-9]+]]:sgpr_128 = S_LOAD_DWORDX4_IMM [[REG_SEQUENCE]], 16, 0 :: ("amdgpu-noclobber" load (<4 x s32>) from %ir.src + 16, addrspace 1)
+ ; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM1]].sub0
+ ; GFX-950-NEXT: [[COPY5:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM1]].sub1
+ ; GFX-950-NEXT: [[COPY6:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM1]].sub2
+ ; GFX-950-NEXT: [[COPY7:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM1]].sub3
+ ; GFX-950-NEXT: [[COPY8:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub0
+ ; GFX-950-NEXT: [[COPY9:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub1
+ ; GFX-950-NEXT: [[COPY10:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub2
+ ; GFX-950-NEXT: [[COPY11:%[0-9]+]]:sreg_32 = COPY [[S_LOAD_DWORDX4_IMM]].sub3
+ ; GFX-950-NEXT: [[REG_SEQUENCE2:%[0-9]+]]:sgpr_224 = REG_SEQUENCE [[COPY8]], %subreg.sub0, [[COPY9]], %subreg.sub1, [[COPY10]], %subreg.sub2, [[COPY11]], %subreg.sub3, [[COPY4]], %subreg.sub4, [[COPY5]], %subreg.sub5, [[COPY6]], %subreg.sub6
+ ; GFX-950-NEXT: [[COPY12:%[0-9]+]]:sgpr_192 = COPY [[REG_SEQUENCE2]].sub0_sub1_sub2_sub3_sub4_sub5, debug-location !4
%val = load i224, ptr addrspace(1) %src
%trunc = trunc i224 %val to i192, !dbg !4
store i192 %trunc, ptr addrspace(1) %dst
@@ -277,8 +279,8 @@ define void @v_trunc_i224_to_i192(ptr addrspace(1) %src, ptr addrspace(1) %dst)
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:vgpr_32 = COPY $vgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:vreg_64_align2 = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX3_:%[0-9]+]]:vreg_96_align2 = GLOBAL_LOAD_DWORDX3 [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (<3 x i32>) from %ir.src + 16, align 8, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX3_:%[0-9]+]]:vreg_96_align2 = GLOBAL_LOAD_DWORDX3 [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (<3 x i32>) from %ir.src + 16, align 16, addrspace 1)
; GFX-950-NEXT: [[COPY4:%[0-9]+]]:vgpr_32 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub0
; GFX-950-NEXT: [[COPY5:%[0-9]+]]:vgpr_32 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub1
; GFX-950-NEXT: [[COPY6:%[0-9]+]]:vgpr_32 = COPY [[GLOBAL_LOAD_DWORDX4_]].sub2
@@ -307,7 +309,7 @@ define amdgpu_ps void @s_trunc_i256_to_i224(ptr addrspace(1) inreg %src, ptr add
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:sreg_32 = COPY $sgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:sreg_32 = COPY $sgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:sreg_64_xexec_xnull = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: early-clobber %20:sgpr_256 = S_LOAD_DWORDX8_IMM_ec [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<8 x i32>) from %ir.src, align 8, addrspace 1)
+ ; GFX-950-NEXT: early-clobber %20:sgpr_256 = S_LOAD_DWORDX8_IMM_ec [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<8 x i32>) from %ir.src, align 16, addrspace 1)
; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sgpr_224 = COPY %20.lo16_hi16_sub1_lo16_sub1_hi16_sub2_lo16_sub2_hi16_sub3_lo16_sub3_hi16_sub4_lo16_sub4_hi16_sub5_lo16_sub5_hi16_sub6_lo16_sub6_hi16, debug-location !4
%val = load i256, ptr addrspace(1) %src
%trunc = trunc i256 %val to i224, !dbg !4
@@ -326,8 +328,8 @@ define void @v_trunc_i256_to_i224(ptr addrspace(1) %src, ptr addrspace(1) %dst)
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:vgpr_32 = COPY $vgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:vreg_64_align2 = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_1:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 16, align 8, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_1:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 16, addrspace 1)
; GFX-950-NEXT: [[REG_SEQUENCE2:%[0-9]+]]:vreg_256_align2 = REG_SEQUENCE [[GLOBAL_LOAD_DWORDX4_]], %subreg.sub0_sub1_sub2_sub3, [[GLOBAL_LOAD_DWORDX4_1]], %subreg.sub4_sub5_sub6_sub7
; GFX-950-NEXT: [[COPY4:%[0-9]+]]:vreg_224_align2 = COPY [[REG_SEQUENCE2]].lo16_hi16_sub1_lo16_sub1_hi16_sub2_lo16_sub2_hi16_sub3_lo16_sub3_hi16_sub4_lo16_sub4_hi16_sub5_lo16_sub5_hi16_sub6_lo16_sub6_hi16, debug-location !4
%val = load i256, ptr addrspace(1) %src
@@ -349,8 +351,8 @@ define amdgpu_ps void @s_trunc_i1024_to_i512(ptr addrspace(1) inreg %src, ptr ad
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:sreg_32 = COPY $sgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:sreg_32 = COPY $sgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:sreg_64_xexec_xnull = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: early-clobber %20:sgpr_512 = S_LOAD_DWORDX16_IMM_ec [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<16 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: early-clobber %23:sgpr_512 = S_LOAD_DWORDX16_IMM_ec [[REG_SEQUENCE]], 64, 0 :: ("amdgpu-noclobber" load (<16 x i32>) from %ir.src + 64, align 8, addrspace 1)
+ ; GFX-950-NEXT: early-clobber %20:sgpr_512 = S_LOAD_DWORDX16_IMM_ec [[REG_SEQUENCE]], 0, 0 :: ("amdgpu-noclobber" load (<16 x i32>) from %ir.src, align 16, addrspace 1)
+ ; GFX-950-NEXT: early-clobber %23:sgpr_512 = S_LOAD_DWORDX16_IMM_ec [[REG_SEQUENCE]], 64, 0 :: ("amdgpu-noclobber" load (<16 x i32>) from %ir.src + 64, align 16, addrspace 1)
; GFX-950-NEXT: [[REG_SEQUENCE2:%[0-9]+]]:sgpr_1024 = REG_SEQUENCE %20, %subreg.sub0_sub1_sub2_sub3_sub4_sub5_sub6_sub7_sub8_sub9_sub10_sub11_sub12_sub13_sub14_sub15, %23, %subreg.sub16_sub17_sub18_sub19_sub20_sub21_sub22_sub23_sub24_sub25_sub26_sub27_sub28_sub29_sub30_sub31
; GFX-950-NEXT: [[COPY4:%[0-9]+]]:sgpr_512 = COPY [[REG_SEQUENCE2]].sub0_sub1_sub2_sub3_sub4_sub5_sub6_sub7_sub8_sub9_sub10_sub11_sub12_sub13_sub14_sub15, debug-location !4
%val = load i1024, ptr addrspace(1) %src
@@ -370,15 +372,15 @@ define void @v_trunc_i1024_to_i512(ptr addrspace(1) %src, ptr addrspace(1) %dst)
; GFX-950-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr2
; GFX-950-NEXT: [[COPY3:%[0-9]+]]:vgpr_32 = COPY $vgpr3
; GFX-950-NEXT: [[REG_SEQUENCE1:%[0-9]+]]:vreg_64_align2 = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY3]], %subreg.sub1
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_1:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 16, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_2:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 32, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 32, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_3:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 48, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 48, align 8, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 0, 0, implicit $exec :: (load (<4 x i32>) from %ir.src, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_1:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 16, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 16, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_2:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 32, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 32, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_3:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 48, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 48, addrspace 1)
; GFX-950-NEXT: [[REG_SEQUENCE2:%[0-9]+]]:vreg_512_align2 = REG_SEQUENCE [[GLOBAL_LOAD_DWORDX4_]], %subreg.sub0_sub1_sub2_sub3, [[GLOBAL_LOAD_DWORDX4_1]], %subreg.sub4_sub5_sub6_sub7, [[GLOBAL_LOAD_DWORDX4_2]], %subreg.sub8_sub9_sub10_sub11, [[GLOBAL_LOAD_DWORDX4_3]], %subreg.sub12_sub13_sub14_sub15
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_4:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 64, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 64, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_5:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 80, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 80, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_6:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 96, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 96, align 8, addrspace 1)
- ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_7:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 112, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 112, align 8, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_4:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 64, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 64, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_5:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 80, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 80, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_6:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 96, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 96, addrspace 1)
+ ; GFX-950-NEXT: [[GLOBAL_LOAD_DWORDX4_7:%[0-9]+]]:vreg_128_align2 = GLOBAL_LOAD_DWORDX4 [[REG_SEQUENCE]], 112, 0, implicit $exec :: (load (<4 x i32>) from %ir.src + 112, addrspace 1)
; GFX-950-NEXT: [[REG_SEQUENCE3:%[0-9]+]]:vreg_512_align2 = REG_SEQUENCE [[GLOBAL_LOAD_DWORDX4_4]], %subreg.sub0_sub1_sub2_sub3, [[GLOBAL_LOAD_DWORDX4_5]], %subreg.sub4_sub5_sub6_sub7, [[GLOBAL_LOAD_DWORDX4_6]], %subreg.sub8_sub9_sub10_sub11, [[GLOBAL_LOAD_DWORDX4_7]], %subreg.sub12_sub13_sub14_sub15
; GFX-950-NEXT: [[REG_SEQUENCE4:%[0-9]+]]:vreg_1024_align2 = REG_SEQUENCE [[REG_SEQUENCE2]], %subreg.sub0_sub1_sub2_sub3_sub4_sub5_sub6_sub7_sub8_sub9_sub10_sub11_sub12_sub13_sub14_sub15, [[REG_SEQUENCE3]], %subreg.sub16_sub17_sub18_sub19_sub20_sub21_sub22_sub23_sub24_sub25_sub26_sub27_sub28_sub29_sub30_sub31
; GFX-950-NEXT: [[COPY4:%[0-9]+]]:vreg_512_align2 = COPY [[REG_SEQUENCE4]].sub0_sub1_sub2_sub3_sub4_sub5_sub6_sub7_sub8_sub9_sub10_sub11_sub12_sub13_sub14_sub15, debug-location !4
diff --git a/llvm/test/CodeGen/AMDGPU/add_i128.ll b/llvm/test/CodeGen/AMDGPU/add_i128.ll
index 5342ec89b0d69..094d7610fb529 100644
--- a/llvm/test/CodeGen/AMDGPU/add_i128.ll
+++ b/llvm/test/CodeGen/AMDGPU/add_i128.ll
@@ -90,7 +90,7 @@ define amdgpu_kernel void @sgpr_operand_reversed(ptr addrspace(1) noalias %out,
define amdgpu_kernel void @test_sreg(ptr addrspace(1) noalias %out, i128 %a, i128 %b) {
; GCN-LABEL: test_sreg:
; GCN: ; %bb.0:
-; GCN-NEXT: s_load_dwordx8 s[8:15], s[4:5], 0xb
+; GCN-NEXT: s_load_dwordx8 s[8:15], s[4:5], 0xd
; GCN-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x9
; GCN-NEXT: s_mov_b32 s3, 0xf000
; GCN-NEXT: s_mov_b32 s2, -1
diff --git a/llvm/test/CodeGen/AMDGPU/amdgpu-attributor-min-agpr-alloc.ll b/llvm/test/CodeGen/AMDGPU/amdgpu-attributor-min-agpr-alloc.ll
index 02eef5ab5e98b..c353a96fb3bfb 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgpu-attributor-min-agpr-alloc.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgpu-attributor-min-agpr-alloc.ll
@@ -764,7 +764,7 @@ define amdgpu_kernel void @kernel_uses_read_register_a56_59(ptr addrspace(1) %pt
; CHECK-LABEL: define amdgpu_kernel void @kernel_uses_read_register_a56_59(
; CHECK-SAME: ptr addrspace(1) [[PTR:%.*]]) #[[ATTR20:[0-9]+]] {
; CHECK-NEXT: [[REG:%.*]] = call i128 @llvm.read_register.i128(metadata [[META3:![0-9]+]])
-; CHECK-NEXT: store i128 [[REG]], ptr addrspace(1) [[PTR]], align 8
+; CHECK-NEXT: store i128 [[REG]], ptr addrspace(1) [[PTR]], align 16
; CHECK-NEXT: call void @use_most()
; CHECK-NEXT: ret void
;
diff --git a/llvm/test/CodeGen/AMDGPU/ctpop64.ll b/llvm/test/CodeGen/AMDGPU/ctpop64.ll
index edf9ef109a42d..2fa36b29506d0 100644
--- a/llvm/test/CodeGen/AMDGPU/ctpop64.ll
+++ b/llvm/test/CodeGen/AMDGPU/ctpop64.ll
@@ -572,7 +572,7 @@ endif:
define amdgpu_kernel void @s_ctpop_i128(ptr addrspace(1) noalias %out, i128 %val) nounwind {
; SI-LABEL: s_ctpop_i128:
; SI: ; %bb.0:
-; SI-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0xb
+; SI-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0xd
; SI-NEXT: s_load_dwordx2 s[4:5], s[4:5], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, -1
@@ -586,7 +586,7 @@ define amdgpu_kernel void @s_ctpop_i128(ptr addrspace(1) noalias %out, i128 %val
;
; VI-LABEL: s_ctpop_i128:
; VI: ; %bb.0:
-; VI-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x2c
+; VI-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x34
; VI-NEXT: s_load_dwordx2 s[4:5], s[4:5], 0x24
; VI-NEXT: s_mov_b32 s7, 0xf000
; VI-NEXT: s_mov_b32 s6, -1
@@ -601,7 +601,7 @@ define amdgpu_kernel void @s_ctpop_i128(ptr addrspace(1) noalias %out, i128 %val
; GFX12-LABEL: s_ctpop_i128:
; GFX12: ; %bb.0:
; GFX12-NEXT: s_clause 0x1
-; GFX12-NEXT: s_load_b128 s[0:3], s[4:5], 0x2c
+; GFX12-NEXT: s_load_b128 s[0:3], s[4:5], 0x34
; GFX12-NEXT: s_load_b64 s[4:5], s[4:5], 0x24
; GFX12-NEXT: v_mov_b32_e32 v1, 0
; GFX12-NEXT: s_wait_kmcnt 0x0
@@ -621,52 +621,51 @@ define amdgpu_kernel void @s_ctpop_i128(ptr addrspace(1) noalias %out, i128 %val
define amdgpu_kernel void @s_ctpop_i65(ptr addrspace(1) noalias %out, i65 %val) nounwind {
; SI-LABEL: s_ctpop_i65:
; SI: ; %bb.0:
-; SI-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x9
-; SI-NEXT: s_load_dword s8, s[4:5], 0xd
-; SI-NEXT: s_mov_b32 s7, 0xf000
-; SI-NEXT: s_mov_b32 s6, -1
+; SI-NEXT: s_load_dword s6, s[4:5], 0xf
+; SI-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x9
+; SI-NEXT: s_load_dwordx2 s[4:5], s[4:5], 0xd
+; SI-NEXT: s_mov_b32 s3, 0xf000
+; SI-NEXT: s_mov_b32 s2, -1
; SI-NEXT: s_waitcnt lgkmcnt(0)
-; SI-NEXT: s_mov_b32 s4, s0
-; SI-NEXT: s_and_b32 s0, s8, 0xff
-; SI-NEXT: s_mov_b32 s5, s1
-; SI-NEXT: s_bcnt1_i32_b32 s0, s0
-; SI-NEXT: s_bcnt1_i32_b64 s1, s[2:3]
-; SI-NEXT: s_add_i32 s0, s1, s0
-; SI-NEXT: v_mov_b32_e32 v0, s0
-; SI-NEXT: buffer_store_dword v0, off, s[4:7], 0
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_bcnt1_i32_b32 s6, s6
+; SI-NEXT: s_bcnt1_i32_b64 s4, s[4:5]
+; SI-NEXT: s_add_i32 s4, s4, s6
+; SI-NEXT: v_mov_b32_e32 v0, s4
+; SI-NEXT: buffer_store_dword v0, off, s[0:3], 0
; SI-NEXT: s_endpgm
;
; VI-LABEL: s_ctpop_i65:
; VI: ; %bb.0:
-; VI-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x24
-; VI-NEXT: s_load_dword s8, s[4:5], 0x34
-; VI-NEXT: s_mov_b32 s7, 0xf000
-; VI-NEXT: s_mov_b32 s6, -1
+; VI-NEXT: s_load_dword s6, s[4:5], 0x3c
+; VI-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x24
+; VI-NEXT: s_load_dwordx2 s[4:5], s[4:5], 0x34
+; VI-NEXT: s_mov_b32 s3, 0xf000
+; VI-NEXT: s_mov_b32 s2, -1
; VI-NEXT: s_waitcnt lgkmcnt(0)
-; VI-NEXT: s_mov_b32 s4, s0
-; VI-NEXT: s_and_b32 s0, s8, 0xff
-; VI-NEXT: s_mov_b32 s5, s1
-; VI-NEXT: s_bcnt1_i32_b32 s0, s0
-; VI-NEXT: s_bcnt1_i32_b64 s1, s[2:3]
-; VI-NEXT: s_add_i32 s0, s1, s0
-; VI-NEXT: v_mov_b32_e32 v0, s0
-; VI-NEXT: buffer_store_dword v0, off, s[4:7], 0
+; VI-NEXT: s_and_b32 s6, s6, 0xff
+; VI-NEXT: s_bcnt1_i32_b32 s6, s6
+; VI-NEXT: s_bcnt1_i32_b64 s4, s[4:5]
+; VI-NEXT: s_add_i32 s4, s4, s6
+; VI-NEXT: v_mov_b32_e32 v0, s4
+; VI-NEXT: buffer_store_dword v0, off, s[0:3], 0
; VI-NEXT: s_endpgm
;
; GFX12-LABEL: s_ctpop_i65:
; GFX12: ; %bb.0:
-; GFX12-NEXT: s_clause 0x1
-; GFX12-NEXT: s_load_u8 s6, s[4:5], 0x34
-; GFX12-NEXT: s_load_b128 s[0:3], s[4:5], 0x24
+; GFX12-NEXT: s_clause 0x2
+; GFX12-NEXT: s_load_u8 s0, s[4:5], 0x3c
+; GFX12-NEXT: s_load_b64 s[2:3], s[4:5], 0x34
+; GFX12-NEXT: s_load_b64 s[4:5], s[4:5], 0x24
; GFX12-NEXT: v_mov_b32_e32 v1, 0
; GFX12-NEXT: s_wait_kmcnt 0x0
-; GFX12-NEXT: s_and_b64 s[4:5], s[6:7], 1
+; GFX12-NEXT: s_and_b64 s[0:1], s[0:1], 1
; GFX12-NEXT: s_bcnt1_i32_b64 s2, s[2:3]
-; GFX12-NEXT: s_bcnt1_i32_b64 s3, s[4:5]
+; GFX12-NEXT: s_bcnt1_i32_b64 s0, s[0:1]
; GFX12-NEXT: s_delay_alu instid0(SALU_CYCLE_1) | instskip(NEXT) | instid1(SALU_CYCLE_1)
-; GFX12-NEXT: s_add_co_i32 s2, s3, s2
-; GFX12-NEXT: v_mov_b32_e32 v0, s2
-; GFX12-NEXT: global_store_b32 v1, v0, s[0:1]
+; GFX12-NEXT: s_add_co_i32 s0, s0, s2
+; GFX12-NEXT: v_mov_b32_e32 v0, s0
+; GFX12-NEXT: global_store_b32 v1, v0, s[4:5]
; GFX12-NEXT: s_endpgm
%ctpop = call i65 @llvm.ctpop.i65(i65 %val) nounwind readnone
%truncctpop = trunc i65 %ctpop to i32
diff --git a/llvm/test/CodeGen/AMDGPU/kernel-args.ll b/llvm/test/CodeGen/AMDGPU/kernel-args.ll
index 97b0651565986..a18148967781a 100644
--- a/llvm/test/CodeGen/AMDGPU/kernel-args.ll
+++ b/llvm/test/CodeGen/AMDGPU/kernel-args.ll
@@ -4459,53 +4459,54 @@ entry:
define amdgpu_kernel void @i65_arg(ptr addrspace(1) nocapture %out, i65 %in) nounwind {
; SI-LABEL: i65_arg:
; SI: ; %bb.0: ; %entry
-; SI-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x9
-; SI-NEXT: s_load_dword s8, s[4:5], 0xd
-; SI-NEXT: s_mov_b32 s7, 0xf000
-; SI-NEXT: s_mov_b32 s6, -1
+; SI-NEXT: s_load_dword s6, s[4:5], 0xf
+; SI-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x9
+; SI-NEXT: s_load_dwordx2 s[4:5], s[4:5], 0xd
+; SI-NEXT: s_mov_b32 s3, 0xf000
+; SI-NEXT: s_mov_b32 s2, -1
; SI-NEXT: s_waitcnt lgkmcnt(0)
-; SI-NEXT: s_mov_b32 s4, s0
-; SI-NEXT: s_and_b32 s0, s8, 1
-; SI-NEXT: s_mov_b32 s5, s1
-; SI-NEXT: v_mov_b32_e32 v0, s0
-; SI-NEXT: buffer_store_byte v0, off, s[4:7], 0 offset:8
+; SI-NEXT: s_and_b32 s6, s6, 1
+; SI-NEXT: v_mov_b32_e32 v0, s6
+; SI-NEXT: buffer_store_byte v0, off, s[0:3], 0 offset:8
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s2
-; SI-NEXT: v_mov_b32_e32 v1, s3
-; SI-NEXT: buffer_store_dwordx2 v[0:1], off, s[4:7], 0
+; SI-NEXT: v_mov_b32_e32 v0, s4
+; SI-NEXT: v_mov_b32_e32 v1, s5
+; SI-NEXT: buffer_store_dwordx2 v[0:1], off, s[0:3], 0
; SI-NEXT: s_endpgm
;
; VI-LABEL: i65_arg:
; VI: ; %bb.0: ; %entry
-; VI-NEXT: s_load_dword s6, s[4:5], 0x34
-; VI-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x24
+; VI-NEXT: s_load_dword s6, s[4:5], 0x3c
+; VI-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x24
+; VI-NEXT: s_load_dwordx2 s[2:3], s[4:5], 0x34
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: s_and_b32 s4, s6, 1
; VI-NEXT: v_mov_b32_e32 v0, s0
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: s_add_u32 s0, s0, 8
; VI-NEXT: s_addc_u32 s1, s1, 0
-; VI-NEXT: v_mov_b32_e32 v6, s4
-; VI-NEXT: v_mov_b32_e32 v5, s1
-; VI-NEXT: v_mov_b32_e32 v4, s0
+; VI-NEXT: v_mov_b32_e32 v4, s4
+; VI-NEXT: v_mov_b32_e32 v3, s1
+; VI-NEXT: v_mov_b32_e32 v2, s0
+; VI-NEXT: flat_store_byte v[2:3], v4
; VI-NEXT: v_mov_b32_e32 v2, s2
; VI-NEXT: v_mov_b32_e32 v3, s3
-; VI-NEXT: flat_store_byte v[4:5], v6
; VI-NEXT: flat_store_dwordx2 v[0:1], v[2:3]
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: i65_arg:
; GFX9: ; %bb.0: ; %entry
-; GFX9-NEXT: s_load_dword s4, s[8:9], 0x10
-; GFX9-NEXT: s_load_dwordx4 s[0:3], s[8:9], 0x0
+; GFX9-NEXT: s_load_dword s4, s[8:9], 0x18
+; GFX9-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x10
+; GFX9-NEXT: s_load_dwordx2 s[2:3], s[8:9], 0x0
; GFX9-NEXT: v_mov_b32_e32 v2, 0
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: s_and_b32 s4, s4, 1
; GFX9-NEXT: v_mov_b32_e32 v3, s4
-; GFX9-NEXT: v_mov_b32_e32 v0, s2
-; GFX9-NEXT: v_mov_b32_e32 v1, s3
-; GFX9-NEXT: global_store_byte v2, v3, s[0:1] offset:8
-; GFX9-NEXT: global_store_dwordx2 v2, v[0:1], s[0:1]
+; GFX9-NEXT: v_mov_b32_e32 v0, s0
+; GFX9-NEXT: v_mov_b32_e32 v1, s1
+; GFX9-NEXT: global_store_byte v2, v3, s[2:3] offset:8
+; GFX9-NEXT: global_store_dwordx2 v2, v[0:1], s[2:3]
; GFX9-NEXT: s_endpgm
;
; EG-LABEL: i65_arg:
@@ -4522,7 +4523,7 @@ define amdgpu_kernel void @i65_arg(ptr addrspace(1) nocapture %out, i65 %in) nou
; EG-NEXT: AND_INT * T1.W, PV.W, literal.x,
; EG-NEXT: 3(4.203895e-45), 0(0.000000e+00)
; EG-NEXT: LSHL T1.W, PV.W, literal.x,
-; EG-NEXT: AND_INT * T2.W, KC0[3].Y, 1,
+; EG-NEXT: AND_INT * T2.W, KC0[3].W, 1,
; EG-NEXT: 3(4.203895e-45), 0(0.000000e+00)
; EG-NEXT: LSHL T1.X, PS, PV.W,
; EG-NEXT: LSHL * T1.W, literal.x, PV.W,
@@ -4533,10 +4534,10 @@ define amdgpu_kernel void @i65_arg(ptr addrspace(1) nocapture %out, i65 %in) nou
; EG-NEXT: ADD_INT * T0.W, KC0[2].Y, literal.y,
; EG-NEXT: 2(2.802597e-45), 4(5.605194e-45)
; EG-NEXT: LSHR T2.X, PV.W, literal.x,
-; EG-NEXT: MOV * T3.X, KC0[3].X,
+; EG-NEXT: MOV * T3.X, KC0[3].Z,
; EG-NEXT: 2(2.802597e-45), 0(0.000000e+00)
; EG-NEXT: LSHR T4.X, KC0[2].Y, literal.x,
-; EG-NEXT: MOV * T5.X, KC0[2].W,
+; EG-NEXT: MOV * T5.X, KC0[3].Y,
; EG-NEXT: 2(2.802597e-45), 0(0.000000e+00)
;
; CM-LABEL: i65_arg:
@@ -4553,7 +4554,7 @@ define amdgpu_kernel void @i65_arg(ptr addrspace(1) nocapture %out, i65 %in) nou
; CM-NEXT: AND_INT * T1.W, PV.W, literal.x,
; CM-NEXT: 3(4.203895e-45), 0(0.000000e+00)
; CM-NEXT: LSHL T0.Z, PV.W, literal.x,
-; CM-NEXT: AND_INT * T1.W, KC0[3].Y, 1,
+; CM-NEXT: AND_INT * T1.W, KC0[3].W, 1,
; CM-NEXT: 3(4.203895e-45), 0(0.000000e+00)
; CM-NEXT: LSHL T1.X, PV.W, PV.Z,
; CM-NEXT: LSHL * T1.W, literal.x, PV.Z,
@@ -4562,12 +4563,12 @@ define amdgpu_kernel void @i65_arg(ptr addrspace(1) nocapture %out, i65 %in) nou
; CM-NEXT: MOV * T1.Z, 0.0,
; CM-NEXT: LSHR * T0.X, KC0[2].Y, literal.x,
; CM-NEXT: 2(2.802597e-45), 0(0.000000e+00)
-; CM-NEXT: MOV T2.X, KC0[2].W,
+; CM-NEXT: MOV T2.X, KC0[3].Y,
; CM-NEXT: ADD_INT * T2.W, KC0[2].Y, literal.x,
; CM-NEXT: 4(5.605194e-45), 0(0.000000e+00)
; CM-NEXT: LSHR * T3.X, PV.W, literal.x,
; CM-NEXT: 2(2.802597e-45), 0(0.000000e+00)
-; CM-NEXT: MOV * T4.X, KC0[3].X,
+; CM-NEXT: MOV * T4.X, KC0[3].Z,
; CM-NEXT: LSHR * T5.X, T0.W, literal.x,
; CM-NEXT: 2(2.802597e-45), 0(0.000000e+00)
entry:
diff --git a/llvm/test/CodeGen/AMDGPU/kernel-argument-dag-lowering.ll b/llvm/test/CodeGen/AMDGPU/kernel-argument-dag-lowering.ll
index e9f4ccda47905..6e12a24927b22 100644
--- a/llvm/test/CodeGen/AMDGPU/kernel-argument-dag-lowering.ll
+++ b/llvm/test/CodeGen/AMDGPU/kernel-argument-dag-lowering.ll
@@ -180,22 +180,23 @@ define amdgpu_kernel void @v6i32_arg(<6 x i32> %in) nounwind {
define amdgpu_kernel void @i65_arg(ptr addrspace(1) nocapture %out, i65 %in) #0 {
; GCN-LABEL: i65_arg:
; GCN: ; %bb.0: ; %entry
-; GCN-NEXT: s_load_dword s4, s[8:9], 0x10
-; GCN-NEXT: s_load_dwordx4 s[0:3], s[8:9], 0x0
+; GCN-NEXT: s_load_dword s4, s[8:9], 0x18
+; GCN-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x10
+; GCN-NEXT: s_load_dwordx2 s[2:3], s[8:9], 0x0
; GCN-NEXT: v_mov_b32_e32 v2, 0
; GCN-NEXT: s_waitcnt lgkmcnt(0)
; GCN-NEXT: s_and_b32 s4, s4, 1
; GCN-NEXT: v_mov_b32_e32 v3, s4
-; GCN-NEXT: v_mov_b32_e32 v0, s2
-; GCN-NEXT: v_mov_b32_e32 v1, s3
-; GCN-NEXT: global_store_byte v2, v3, s[0:1] offset:8
-; GCN-NEXT: global_store_dwordx2 v2, v[0:1], s[0:1]
+; GCN-NEXT: v_mov_b32_e32 v0, s0
+; GCN-NEXT: v_mov_b32_e32 v1, s1
+; GCN-NEXT: global_store_byte v2, v3, s[2:3] offset:8
+; GCN-NEXT: global_store_dwordx2 v2, v[0:1], s[2:3]
; GCN-NEXT: s_endpgm
entry:
store i65 %in, ptr addrspace(1) %out, align 4
ret void
}
-; GCN: .amdhsa_kernarg_size 24
+; GCN: .amdhsa_kernarg_size 32
define amdgpu_kernel void @empty_struct_arg({} %in) #0 {
; GCN-LABEL: empty_struct_arg:
diff --git a/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-constants.ll b/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-constants.ll
index e471178394303..970535c96745f 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-constants.ll
+++ b/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-constants.ll
@@ -117,7 +117,7 @@ define ptr @gep_of_p7_struct() {
define ptr addrspace(7) @gep_p7_from_p7() {
; CHECK-LABEL: define { ptr addrspace(8), i32 } @gep_p7_from_p7() {
-; CHECK-NEXT: ret { ptr addrspace(8), i32 } { ptr addrspace(8) @buf, i32 48 }
+; CHECK-NEXT: ret { ptr addrspace(8), i32 } { ptr addrspace(8) @buf, i32 64 }
;
ret ptr addrspace(7) getelementptr (ptr addrspace(7),
ptr addrspace(7) addrspacecast (ptr addrspace(8) @buf to ptr addrspace(7)),
diff --git a/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-contents-legalization.ll b/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-contents-legalization.ll
index 5efed9e569f2f..91b0c643c6e23 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-contents-legalization.ll
+++ b/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-contents-legalization.ll
@@ -100,7 +100,7 @@ define void @store_i64(i64 %data, ptr addrspace(8) inreg %buf) {
define i128 @load_i128(ptr addrspace(8) inreg %buf) {
; CHECK-LABEL: define i128 @load_i128(
; CHECK-SAME: ptr addrspace(8) inreg [[BUF:%.*]]) {
-; CHECK-NEXT: [[RET:%.*]] = call i128 @llvm.amdgcn.raw.ptr.buffer.load.i128(ptr addrspace(8) align 8 [[BUF]], i32 0, i32 0, i32 0)
+; CHECK-NEXT: [[RET:%.*]] = call i128 @llvm.amdgcn.raw.ptr.buffer.load.i128(ptr addrspace(8) align 16 [[BUF]], i32 0, i32 0, i32 0)
; CHECK-NEXT: ret i128 [[RET]]
;
%p = addrspacecast ptr addrspace(8) %buf to ptr addrspace(7)
@@ -111,7 +111,7 @@ define i128 @load_i128(ptr addrspace(8) inreg %buf) {
define void @store_i128(i128 %data, ptr addrspace(8) inreg %buf) {
; CHECK-LABEL: define void @store_i128(
; CHECK-SAME: i128 [[DATA:%.*]], ptr addrspace(8) inreg [[BUF:%.*]]) {
-; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.i128(i128 [[DATA]], ptr addrspace(8) align 8 [[BUF]], i32 0, i32 0, i32 0)
+; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.i128(i128 [[DATA]], ptr addrspace(8) align 16 [[BUF]], i32 0, i32 0, i32 0)
; CHECK-NEXT: ret void
;
%p = addrspacecast ptr addrspace(8) %buf to ptr addrspace(7)
@@ -1720,7 +1720,7 @@ define void @store_i40(i40 %data, ptr addrspace(8) inreg %buf) {
define i96 @load_i96(ptr addrspace(8) inreg %buf) {
; CHECK-LABEL: define i96 @load_i96(
; CHECK-SAME: ptr addrspace(8) inreg [[BUF:%.*]]) {
-; CHECK-NEXT: [[RET_LOADABLE:%.*]] = call <3 x i32> @llvm.amdgcn.raw.ptr.buffer.load.v3i32(ptr addrspace(8) align 8 [[BUF]], i32 0, i32 0, i32 0)
+; CHECK-NEXT: [[RET_LOADABLE:%.*]] = call <3 x i32> @llvm.amdgcn.raw.ptr.buffer.load.v3i32(ptr addrspace(8) align 16 [[BUF]], i32 0, i32 0, i32 0)
; CHECK-NEXT: [[RET:%.*]] = bitcast <3 x i32> [[RET_LOADABLE]] to i96
; CHECK-NEXT: ret i96 [[RET]]
;
@@ -1733,7 +1733,7 @@ define void @store_i96(i96 %data, ptr addrspace(8) inreg %buf) {
; CHECK-LABEL: define void @store_i96(
; CHECK-SAME: i96 [[DATA:%.*]], ptr addrspace(8) inreg [[BUF:%.*]]) {
; CHECK-NEXT: [[DATA_LEGAL:%.*]] = bitcast i96 [[DATA]] to <3 x i32>
-; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.v3i32(<3 x i32> [[DATA_LEGAL]], ptr addrspace(8) align 8 [[BUF]], i32 0, i32 0, i32 0)
+; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.v3i32(<3 x i32> [[DATA_LEGAL]], ptr addrspace(8) align 16 [[BUF]], i32 0, i32 0, i32 0)
; CHECK-NEXT: ret void
;
%p = addrspacecast ptr addrspace(8) %buf to ptr addrspace(7)
@@ -1744,10 +1744,10 @@ define void @store_i96(i96 %data, ptr addrspace(8) inreg %buf) {
define i160 @load_i160(ptr addrspace(8) inreg %buf) {
; CHECK-LABEL: define i160 @load_i160(
; CHECK-SAME: ptr addrspace(8) inreg [[BUF:%.*]]) {
-; CHECK-NEXT: [[RET_OFF_0:%.*]] = call <4 x i32> @llvm.amdgcn.raw.ptr.buffer.load.v4i32(ptr addrspace(8) align 8 [[BUF]], i32 0, i32 0, i32 0)
+; CHECK-NEXT: [[RET_OFF_0:%.*]] = call <4 x i32> @llvm.amdgcn.raw.ptr.buffer.load.v4i32(ptr addrspace(8) align 16 [[BUF]], i32 0, i32 0, i32 0)
; CHECK-NEXT: [[RET_EXT_0:%.*]] = shufflevector <4 x i32> [[RET_OFF_0]], <4 x i32> poison, <5 x i32> <i32 0, i32 1, i32 2, i32 3, i32 poison>
; CHECK-NEXT: [[RET_PARTS_0:%.*]] = shufflevector <5 x i32> poison, <5 x i32> [[RET_EXT_0]], <5 x i32> <i32 5, i32 6, i32 7, i32 8, i32 4>
-; CHECK-NEXT: [[RET_OFF_16:%.*]] = call i32 @llvm.amdgcn.raw.ptr.buffer.load.i32(ptr addrspace(8) align 8 [[BUF]], i32 16, i32 0, i32 0)
+; CHECK-NEXT: [[RET_OFF_16:%.*]] = call i32 @llvm.amdgcn.raw.ptr.buffer.load.i32(ptr addrspace(8) align 16 [[BUF]], i32 16, i32 0, i32 0)
; CHECK-NEXT: [[RET_SLICE_4:%.*]] = insertelement <5 x i32> [[RET_PARTS_0]], i32 [[RET_OFF_16]], i64 4
; CHECK-NEXT: [[RET:%.*]] = bitcast <5 x i32> [[RET_SLICE_4]] to i160
; CHECK-NEXT: ret i160 [[RET]]
@@ -1762,9 +1762,9 @@ define void @store_i160(i160 %data, ptr addrspace(8) inreg %buf) {
; CHECK-SAME: i160 [[DATA:%.*]], ptr addrspace(8) inreg [[BUF:%.*]]) {
; CHECK-NEXT: [[DATA_LEGAL:%.*]] = bitcast i160 [[DATA]] to <5 x i32>
; CHECK-NEXT: [[DATA_SLICE_0:%.*]] = shufflevector <5 x i32> [[DATA_LEGAL]], <5 x i32> poison, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
-; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.v4i32(<4 x i32> [[DATA_SLICE_0]], ptr addrspace(8) align 8 [[BUF]], i32 0, i32 0, i32 0)
+; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.v4i32(<4 x i32> [[DATA_SLICE_0]], ptr addrspace(8) align 16 [[BUF]], i32 0, i32 0, i32 0)
; CHECK-NEXT: [[DATA_SLICE_4:%.*]] = extractelement <5 x i32> [[DATA_LEGAL]], i64 4
-; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.i32(i32 [[DATA_SLICE_4]], ptr addrspace(8) align 8 [[BUF]], i32 16, i32 0, i32 0)
+; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.i32(i32 [[DATA_SLICE_4]], ptr addrspace(8) align 16 [[BUF]], i32 16, i32 0, i32 0)
; CHECK-NEXT: ret void
;
%p = addrspacecast ptr addrspace(8) %buf to ptr addrspace(7)
@@ -1775,10 +1775,10 @@ define void @store_i160(i160 %data, ptr addrspace(8) inreg %buf) {
define i256 @load_i256(ptr addrspace(8) inreg %buf) {
; CHECK-LABEL: define i256 @load_i256(
; CHECK-SAME: ptr addrspace(8) inreg [[BUF:%.*]]) {
-; CHECK-NEXT: [[RET_OFF_0:%.*]] = call <4 x i32> @llvm.amdgcn.raw.ptr.buffer.load.v4i32(ptr addrspace(8) align 8 [[BUF]], i32 0, i32 0, i32 0)
+; CHECK-NEXT: [[RET_OFF_0:%.*]] = call <4 x i32> @llvm.amdgcn.raw.ptr.buffer.load.v4i32(ptr addrspace(8) align 16 [[BUF]], i32 0, i32 0, i32 0)
; CHECK-NEXT: [[RET_EXT_0:%.*]] = shufflevector <4 x i32> [[RET_OFF_0]], <4 x i32> poison, <8 x i32> <i32 0, i32 1, i32 2, i32 3, i32 poison, i32 poison, i32 poison, i32 poison>
; CHECK-NEXT: [[RET_PARTS_0:%.*]] = shufflevector <8 x i32> poison, <8 x i32> [[RET_EXT_0]], <8 x i32> <i32 8, i32 9, i32 10, i32 11, i32 4, i32 5, i32 6, i32 7>
-; CHECK-NEXT: [[RET_OFF_16:%.*]] = call <4 x i32> @llvm.amdgcn.raw.ptr.buffer.load.v4i32(ptr addrspace(8) align 8 [[BUF]], i32 16, i32 0, i32 0)
+; CHECK-NEXT: [[RET_OFF_16:%.*]] = call <4 x i32> @llvm.amdgcn.raw.ptr.buffer.load.v4i32(ptr addrspace(8) align 16 [[BUF]], i32 16, i32 0, i32 0)
; CHECK-NEXT: [[RET_EXT_4:%.*]] = shufflevector <4 x i32> [[RET_OFF_16]], <4 x i32> poison, <8 x i32> <i32 0, i32 1, i32 2, i32 3, i32 poison, i32 poison, i32 poison, i32 poison>
; CHECK-NEXT: [[RET_PARTS_4:%.*]] = shufflevector <8 x i32> [[RET_PARTS_0]], <8 x i32> [[RET_EXT_4]], <8 x i32> <i32 0, i32 1, i32 2, i32 3, i32 8, i32 9, i32 10, i32 11>
; CHECK-NEXT: [[RET:%.*]] = bitcast <8 x i32> [[RET_PARTS_4]] to i256
@@ -1794,9 +1794,9 @@ define void @store_i256(i256 %data, ptr addrspace(8) inreg %buf) {
; CHECK-SAME: i256 [[DATA:%.*]], ptr addrspace(8) inreg [[BUF:%.*]]) {
; CHECK-NEXT: [[DATA_LEGAL:%.*]] = bitcast i256 [[DATA]] to <8 x i32>
; CHECK-NEXT: [[DATA_SLICE_0:%.*]] = shufflevector <8 x i32> [[DATA_LEGAL]], <8 x i32> poison, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
-; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.v4i32(<4 x i32> [[DATA_SLICE_0]], ptr addrspace(8) align 8 [[BUF]], i32 0, i32 0, i32 0)
+; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.v4i32(<4 x i32> [[DATA_SLICE_0]], ptr addrspace(8) align 16 [[BUF]], i32 0, i32 0, i32 0)
; CHECK-NEXT: [[DATA_SLICE_4:%.*]] = shufflevector <8 x i32> [[DATA_LEGAL]], <8 x i32> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
-; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.v4i32(<4 x i32> [[DATA_SLICE_4]], ptr addrspace(8) align 8 [[BUF]], i32 16, i32 0, i32 0)
+; CHECK-NEXT: call void @llvm.amdgcn.raw.ptr.buffer.store.v4i32(<4 x i32> [[DATA_SLICE_4]], ptr addrspace(8) align 16 [[BUF]], i32 16, i32 0, i32 0)
; CHECK-NEXT: ret void
;
%p = addrspacecast ptr addrspace(8) %buf to ptr addrspace(7)
diff --git a/llvm/test/CodeGen/AMDGPU/mul.ll b/llvm/test/CodeGen/AMDGPU/mul.ll
index 97aec3a73ec2f..b7d0e44ba1af4 100644
--- a/llvm/test/CodeGen/AMDGPU/mul.ll
+++ b/llvm/test/CodeGen/AMDGPU/mul.ll
@@ -3393,8 +3393,8 @@ endif:
define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a, [8 x i32], i128 %b) nounwind #0 {
; SI-LABEL: s_mul_i128:
; SI: ; %bb.0: ; %entry
-; SI-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x13
-; SI-NEXT: s_load_dwordx4 s[12:15], s[4:5], 0x1f
+; SI-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x15
+; SI-NEXT: s_load_dwordx4 s[12:15], s[4:5], 0x21
; SI-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x9
; SI-NEXT: s_mov_b32 s3, 0xf000
; SI-NEXT: s_mov_b32 s2, -1
@@ -3442,8 +3442,8 @@ define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a,
;
; VI-LABEL: s_mul_i128:
; VI: ; %bb.0: ; %entry
-; VI-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x4c
-; VI-NEXT: s_load_dwordx4 s[12:15], s[4:5], 0x7c
+; VI-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x54
+; VI-NEXT: s_load_dwordx4 s[12:15], s[4:5], 0x84
; VI-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x24
; VI-NEXT: s_mov_b32 s3, 0xf000
; VI-NEXT: s_mov_b32 s2, -1
@@ -3477,8 +3477,8 @@ define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a,
;
; GFX9-LABEL: s_mul_i128:
; GFX9: ; %bb.0: ; %entry
-; GFX9-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x7c
-; GFX9-NEXT: s_load_dwordx4 s[12:15], s[4:5], 0x4c
+; GFX9-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x84
+; GFX9-NEXT: s_load_dwordx4 s[12:15], s[4:5], 0x54
; GFX9-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x24
; GFX9-NEXT: s_mov_b32 s3, 0xf000
; GFX9-NEXT: s_mov_b32 s2, -1
@@ -3528,8 +3528,8 @@ define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a,
; GFX10-LABEL: s_mul_i128:
; GFX10: ; %bb.0: ; %entry
; GFX10-NEXT: s_clause 0x2
-; GFX10-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x4c
-; GFX10-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x7c
+; GFX10-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x54
+; GFX10-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x84
; GFX10-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x24
; GFX10-NEXT: s_mov_b32 s6, 0
; GFX10-NEXT: s_mov_b32 s5, s6
@@ -3579,8 +3579,8 @@ define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a,
; GFX11-LABEL: s_mul_i128:
; GFX11: ; %bb.0: ; %entry
; GFX11-NEXT: s_clause 0x2
-; GFX11-NEXT: s_load_b128 s[0:3], s[4:5], 0x4c
-; GFX11-NEXT: s_load_b128 s[8:11], s[4:5], 0x7c
+; GFX11-NEXT: s_load_b128 s[0:3], s[4:5], 0x54
+; GFX11-NEXT: s_load_b128 s[8:11], s[4:5], 0x84
; GFX11-NEXT: s_load_b64 s[4:5], s[4:5], 0x24
; GFX11-NEXT: s_mov_b32 s6, 0
; GFX11-NEXT: s_delay_alu instid0(SALU_CYCLE_1)
@@ -3630,8 +3630,8 @@ define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a,
; GFX12-LABEL: s_mul_i128:
; GFX12: ; %bb.0: ; %entry
; GFX12-NEXT: s_clause 0x1
-; GFX12-NEXT: s_load_b128 s[8:11], s[4:5], 0x7c
-; GFX12-NEXT: s_load_b128 s[12:15], s[4:5], 0x4c
+; GFX12-NEXT: s_load_b128 s[8:11], s[4:5], 0x84
+; GFX12-NEXT: s_load_b128 s[12:15], s[4:5], 0x54
; GFX12-NEXT: s_mov_b32 s3, 0
; GFX12-NEXT: s_load_b64 s[0:1], s[4:5], 0x24
; GFX12-NEXT: s_mov_b32 s7, s3
@@ -3676,8 +3676,8 @@ define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a,
; GFX1250-NEXT: v_nop
; GFX1250-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_WAVE_MODE, 25, 1), 1 ; msbs: dst=0 src0=0 src1=0 src2=0
; GFX1250-NEXT: s_clause 0x2
-; GFX1250-NEXT: s_load_b128 s[8:11], s[4:5], 0x7c nv
-; GFX1250-NEXT: s_load_b128 s[12:15], s[4:5], 0x4c nv
+; GFX1250-NEXT: s_load_b128 s[8:11], s[4:5], 0x84 nv
+; GFX1250-NEXT: s_load_b128 s[12:15], s[4:5], 0x54 nv
; GFX1250-NEXT: s_load_b64 s[0:1], s[4:5], 0x24 nv
; GFX1250-NEXT: s_wait_xcnt 0x0
; GFX1250-NEXT: s_mov_b64 s[4:5], 0xffffffff
@@ -3722,8 +3722,8 @@ define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a,
; GFX13-LABEL: s_mul_i128:
; GFX13: ; %bb.0: ; %entry
; GFX13-NEXT: s_clause 0x2
-; GFX13-NEXT: s_load_b128 s[8:11], s[4:5], 0x7c nv
-; GFX13-NEXT: s_load_b128 s[12:15], s[4:5], 0x4c nv
+; GFX13-NEXT: s_load_b128 s[8:11], s[4:5], 0x84 nv
+; GFX13-NEXT: s_load_b128 s[12:15], s[4:5], 0x54 nv
; GFX13-NEXT: s_load_b64 s[0:1], s[4:5], 0x24 nv
; GFX13-NEXT: s_mov_b64 s[4:5], 0xffffffff
; GFX13-NEXT: s_mov_b32 s3, 0
@@ -3770,32 +3770,32 @@ define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a,
; EG-NEXT: CF_END
; EG-NEXT: PAD
; EG-NEXT: ALU clause starting at 4:
-; EG-NEXT: MULLO_INT * T0.X, KC0[5].X, KC0[8].X,
-; EG-NEXT: MULHI * T0.Y, KC0[5].X, KC0[8].X,
-; EG-NEXT: MULLO_INT * T0.Z, KC0[8].Y, KC0[4].W,
-; EG-NEXT: MULLO_INT * T0.W, KC0[8].X, KC0[5].Y,
-; EG-NEXT: MULHI * T1.X, KC0[5].X, KC0[7].W,
-; EG-NEXT: MULHI * T1.Y, KC0[4].W, KC0[8].X,
-; EG-NEXT: MULHI * T1.Z, KC0[8].Y, KC0[4].W,
-; EG-NEXT: MULLO_INT * T1.W, KC0[8].Y, KC0[5].X,
-; EG-NEXT: MULHI * T2.X, KC0[7].W, KC0[5].Y,
-; EG-NEXT: MULLO_INT * T2.Y, KC0[5].X, KC0[7].W,
-; EG-NEXT: MULHI * T2.Z, KC0[4].W, KC0[7].W,
+; EG-NEXT: MULLO_INT * T0.X, KC0[5].Z, KC0[8].Z,
+; EG-NEXT: MULHI * T0.Y, KC0[5].Z, KC0[8].Z,
+; EG-NEXT: MULLO_INT * T0.Z, KC0[8].W, KC0[5].Y,
+; EG-NEXT: MULLO_INT * T0.W, KC0[8].Z, KC0[5].W,
+; EG-NEXT: MULHI * T1.X, KC0[5].Z, KC0[8].Y,
+; EG-NEXT: MULHI * T1.Y, KC0[5].Y, KC0[8].Z,
+; EG-NEXT: MULHI * T1.Z, KC0[8].W, KC0[5].Y,
+; EG-NEXT: MULLO_INT * T1.W, KC0[8].W, KC0[5].Z,
+; EG-NEXT: MULHI * T2.X, KC0[8].Y, KC0[5].W,
+; EG-NEXT: MULLO_INT * T2.Y, KC0[5].Z, KC0[8].Y,
+; EG-NEXT: MULHI * T2.Z, KC0[5].Y, KC0[8].Y,
; EG-NEXT: ADD_INT T2.W, T2.Y, PS,
-; EG-NEXT: MULLO_INT * T3.X, KC0[4].W, KC0[8].X,
+; EG-NEXT: MULLO_INT * T3.X, KC0[5].Y, KC0[8].Z,
; EG-NEXT: ADDC_UINT T2.Z, T2.Y, T2.Z,
; EG-NEXT: ADDC_UINT T3.W, PS, PV.W,
-; EG-NEXT: MULLO_INT * T2.Y, KC0[7].W, KC0[5].Z,
+; EG-NEXT: MULLO_INT * T2.Y, KC0[8].Y, KC0[6].X,
; EG-NEXT: ADD_INT T2.X, T2.X, PS,
; EG-NEXT: ADD_INT T2.Y, T1.Z, T1.W,
; EG-NEXT: ADD_INT T1.Z, T1.Y, PV.W,
; EG-NEXT: ADD_INT T1.W, T1.X, PV.Z, BS:VEC_120/SCL_212
-; EG-NEXT: MULLO_INT * T1.X, KC0[8].Z, KC0[4].W,
+; EG-NEXT: MULLO_INT * T1.X, KC0[9].X, KC0[5].Y,
; EG-NEXT: ADD_INT T4.X, PV.W, PV.Z,
; EG-NEXT: ADDC_UINT T1.Y, PV.W, PV.Z,
; EG-NEXT: ADD_INT T1.Z, PV.Y, PS,
; EG-NEXT: ADD_INT T0.W, PV.X, T0.W,
-; EG-NEXT: MULLO_INT * T1.X, KC0[7].W, KC0[5].Y,
+; EG-NEXT: MULLO_INT * T1.X, KC0[8].Y, KC0[5].W,
; EG-NEXT: ADD_INT T2.Y, PV.Z, PV.W,
; EG-NEXT: ADDC_UINT T1.Z, T0.Z, PS,
; EG-NEXT: ADD_INT T0.W, T0.Y, PV.Y,
@@ -3811,7 +3811,7 @@ define amdgpu_kernel void @s_mul_i128(ptr addrspace(1) %out, [8 x i32], i128 %a,
; EG-NEXT: ADD_INT * T0.Y, T3.X, T2.W,
; EG-NEXT: LSHR * T1.X, KC0[2].Y, literal.x,
; EG-NEXT: 2(2.802597e-45), 0(0.000000e+00)
-; EG-NEXT: MULLO_INT * T0.X, KC0[4].W, KC0[7].W,
+; EG-NEXT: MULLO_INT * T0.X, KC0[5].Y, KC0[8].Y,
entry:
%mul = mul i128 %a, %b
store i128 %mul, ptr addrspace(1) %out
diff --git a/llvm/test/CodeGen/AMDGPU/opencl-printf.ll b/llvm/test/CodeGen/AMDGPU/opencl-printf.ll
index 292fbe2331b95..ac0dce92df408 100644
--- a/llvm/test/CodeGen/AMDGPU/opencl-printf.ll
+++ b/llvm/test/CodeGen/AMDGPU/opencl-printf.ll
@@ -292,9 +292,9 @@ define amdgpu_kernel void @format_str_d(i1 %i1, i4 %i4, i8 %i8, i24 %i24, i16 %i
; GCN-NEXT: [[PRINTBUFFNEXTPTR5:%.*]] = getelementptr i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR4]], i32 4
; GCN-NEXT: store i64 [[I64:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR5]], align 8
; GCN-NEXT: [[PRINTBUFFNEXTPTR6:%.*]] = getelementptr i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR5]], i32 8
-; GCN-NEXT: store i96 [[I96:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR6]], align 8
+; GCN-NEXT: store i96 [[I96:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR6]], align 16
; GCN-NEXT: [[PRINTBUFFNEXTPTR7:%.*]] = getelementptr i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR6]], i32 16
-; GCN-NEXT: store i128 [[I128:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR7]], align 8
+; GCN-NEXT: store i128 [[I128:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR7]], align 16
; GCN-NEXT: [[PRINTBUFFNEXTPTR8:%.*]] = getelementptr i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR7]], i32 16
; GCN-NEXT: store i32 1234, ptr addrspace(1) [[PRINTBUFFNEXTPTR8]], align 4
; GCN-NEXT: br label [[TMP7]]
@@ -335,9 +335,9 @@ define amdgpu_kernel void @format_str_u(i1 %i1, i4 %i4, i8 %i8, i24 %i24, i16 %i
; GCN-NEXT: [[PRINTBUFFNEXTPTR5:%.*]] = getelementptr i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR4]], i32 4
; GCN-NEXT: store i64 [[I64:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR5]], align 8
; GCN-NEXT: [[PRINTBUFFNEXTPTR6:%.*]] = getelementptr i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR5]], i32 8
-; GCN-NEXT: store i96 [[I96:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR6]], align 8
+; GCN-NEXT: store i96 [[I96:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR6]], align 16
; GCN-NEXT: [[PRINTBUFFNEXTPTR7:%.*]] = getelementptr i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR6]], i32 16
-; GCN-NEXT: store i128 [[I128:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR7]], align 8
+; GCN-NEXT: store i128 [[I128:%.*]], ptr addrspace(1) [[PRINTBUFFNEXTPTR7]], align 16
; GCN-NEXT: [[PRINTBUFFNEXTPTR8:%.*]] = getelementptr i8, ptr addrspace(1) [[PRINTBUFFNEXTPTR7]], i32 16
; GCN-NEXT: store i32 1234, ptr addrspace(1) [[PRINTBUFFNEXTPTR8]], align 4
; GCN-NEXT: br label [[TMP7]]
diff --git a/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs-IR-lowering.ll b/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs-IR-lowering.ll
index 86b17eac16344..fd8b44f138d08 100644
--- a/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs-IR-lowering.ll
+++ b/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs-IR-lowering.ll
@@ -30,7 +30,7 @@ define amdgpu_kernel void @preload_block_count_x(ptr addrspace(1) %out) {
define amdgpu_kernel void @no_free_sgprs_block_count_x(ptr addrspace(1) %out, i512) {
; NO-PRELOAD-LABEL: define amdgpu_kernel void @no_free_sgprs_block_count_x(
; NO-PRELOAD-SAME: ptr addrspace(1) [[OUT:%.*]], i512 [[TMP0:%.*]]) #[[ATTR0]] {
-; NO-PRELOAD-NEXT: [[NO_FREE_SGPRS_BLOCK_COUNT_X_KERNARG_SEGMENT:%.*]] = call nonnull align 16 dereferenceable(328) ptr addrspace(4) @llvm.amdgcn.kernarg.segment.ptr()
+; NO-PRELOAD-NEXT: [[NO_FREE_SGPRS_BLOCK_COUNT_X_KERNARG_SEGMENT:%.*]] = call nonnull align 16 dereferenceable(336) ptr addrspace(4) @llvm.amdgcn.kernarg.segment.ptr()
; NO-PRELOAD-NEXT: [[OUT_KERNARG_OFFSET:%.*]] = getelementptr inbounds i8, ptr addrspace(4) [[NO_FREE_SGPRS_BLOCK_COUNT_X_KERNARG_SEGMENT]], i64 0
; NO-PRELOAD-NEXT: [[OUT_LOAD:%.*]] = load ptr addrspace(1), ptr addrspace(4) [[OUT_KERNARG_OFFSET]], align 16, !invariant.load [[META0]]
; NO-PRELOAD-NEXT: [[IMP_ARG_PTR:%.*]] = call ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr()
@@ -40,7 +40,7 @@ define amdgpu_kernel void @no_free_sgprs_block_count_x(ptr addrspace(1) %out, i5
;
; PRELOAD-LABEL: define amdgpu_kernel void @no_free_sgprs_block_count_x(
; PRELOAD-SAME: ptr addrspace(1) inreg [[OUT:%.*]], i512 [[TMP0:%.*]]) #[[ATTR0]] {
-; PRELOAD-NEXT: [[NO_FREE_SGPRS_BLOCK_COUNT_X_KERNARG_SEGMENT:%.*]] = call nonnull align 16 dereferenceable(328) ptr addrspace(4) @llvm.amdgcn.kernarg.segment.ptr()
+; PRELOAD-NEXT: [[NO_FREE_SGPRS_BLOCK_COUNT_X_KERNARG_SEGMENT:%.*]] = call nonnull align 16 dereferenceable(336) ptr addrspace(4) @llvm.amdgcn.kernarg.segment.ptr()
; PRELOAD-NEXT: [[IMP_ARG_PTR:%.*]] = call ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr()
; PRELOAD-NEXT: [[LOAD:%.*]] = load i32, ptr addrspace(4) [[IMP_ARG_PTR]], align 4
; PRELOAD-NEXT: store i32 [[LOAD]], ptr addrspace(1) [[OUT]], align 4
diff --git a/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs.ll b/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs.ll
index 17dd254a9bf33..e84a682d2e678 100644
--- a/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs.ll
+++ b/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs.ll
@@ -100,7 +100,7 @@ define amdgpu_kernel void @no_free_sgprs_block_count_x(ptr addrspace(1) inreg %o
; GFX942-NEXT: .p2align 8
; GFX942-NEXT: ; %bb.2:
; GFX942-NEXT: .LBB2_0:
-; GFX942-NEXT: s_load_dword s0, s[4:5], 0x28
+; GFX942-NEXT: s_load_dword s0, s[4:5], 0x30
; GFX942-NEXT: v_mov_b32_e32 v0, 0
; GFX942-NEXT: s_waitcnt lgkmcnt(0)
; GFX942-NEXT: v_mov_b32_e32 v1, s0
@@ -115,7 +115,7 @@ define amdgpu_kernel void @no_free_sgprs_block_count_x(ptr addrspace(1) inreg %o
; GFX90a-NEXT: .p2align 8
; GFX90a-NEXT: ; %bb.2:
; GFX90a-NEXT: .LBB2_0:
-; GFX90a-NEXT: s_load_dword s0, s[8:9], 0x28
+; GFX90a-NEXT: s_load_dword s0, s[8:9], 0x30
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -127,7 +127,7 @@ define amdgpu_kernel void @no_free_sgprs_block_count_x(ptr addrspace(1) inreg %o
; GFX1250-NEXT: global_prefetch_b8 v0, null scope:SCOPE_SE
; GFX1250-NEXT: v_nop
; GFX1250-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_WAVE_MODE, 25, 1), 1 ; msbs: dst=0 src0=0 src1=0 src2=0
-; GFX1250-NEXT: v_dual_mov_b32 v0, 0 :: v_dual_mov_b32 v1, s18
+; GFX1250-NEXT: v_dual_mov_b32 v0, 0 :: v_dual_mov_b32 v1, s20
; GFX1250-NEXT: global_store_b32 v0, v1, s[8:9]
; GFX1250-NEXT: s_endpgm
%imp_arg_ptr = call ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr()
@@ -1044,14 +1044,17 @@ define amdgpu_kernel void @preload_block_max_user_sgprs(ptr addrspace(1) inreg %
; GFX942: ; %bb.1:
; GFX942-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
; GFX942-NEXT: s_load_dwordx8 s[4:11], s[0:1], 0x8
-; GFX942-NEXT: s_load_dword s12, s[0:1], 0x28
+; GFX942-NEXT: s_load_dwordx2 s[12:13], s[0:1], 0x28
+; GFX942-NEXT: s_load_dword s14, s[0:1], 0x30
; GFX942-NEXT: s_waitcnt lgkmcnt(0)
; GFX942-NEXT: s_branch .LBB21_0
; GFX942-NEXT: .p2align 8
; GFX942-NEXT: ; %bb.2:
; GFX942-NEXT: .LBB21_0:
+; GFX942-NEXT: s_load_dword s0, s[0:1], 0x38
; GFX942-NEXT: v_mov_b32_e32 v0, 0
-; GFX942-NEXT: v_mov_b32_e32 v1, s12
+; GFX942-NEXT: s_waitcnt lgkmcnt(0)
+; GFX942-NEXT: v_mov_b32_e32 v1, s0
; GFX942-NEXT: global_store_dword v0, v1, s[2:3]
; GFX942-NEXT: s_endpgm
;
@@ -1063,7 +1066,7 @@ define amdgpu_kernel void @preload_block_max_user_sgprs(ptr addrspace(1) inreg %
; GFX90a-NEXT: .p2align 8
; GFX90a-NEXT: ; %bb.2:
; GFX90a-NEXT: .LBB21_0:
-; GFX90a-NEXT: s_load_dword s0, s[4:5], 0x28
+; GFX90a-NEXT: s_load_dword s0, s[4:5], 0x38
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -1075,7 +1078,7 @@ define amdgpu_kernel void @preload_block_max_user_sgprs(ptr addrspace(1) inreg %
; GFX1250-NEXT: global_prefetch_b8 v0, null scope:SCOPE_SE
; GFX1250-NEXT: v_nop
; GFX1250-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_WAVE_MODE, 25, 1), 1 ; msbs: dst=0 src0=0 src1=0 src2=0
-; GFX1250-NEXT: v_dual_mov_b32 v0, 0 :: v_dual_mov_b32 v1, s12
+; GFX1250-NEXT: v_dual_mov_b32 v0, 0 :: v_dual_mov_b32 v1, s16
; GFX1250-NEXT: global_store_b32 v0, v1, s[2:3]
; GFX1250-NEXT: s_endpgm
%imp_arg_ptr = call ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr()
diff --git a/llvm/test/CodeGen/AMDGPU/store-weird-sizes.ll b/llvm/test/CodeGen/AMDGPU/store-weird-sizes.ll
index 5fd6b75662efb..82abb64c218c2 100644
--- a/llvm/test/CodeGen/AMDGPU/store-weird-sizes.ll
+++ b/llvm/test/CodeGen/AMDGPU/store-weird-sizes.ll
@@ -252,9 +252,9 @@ define amdgpu_kernel void @local_store_i48(ptr addrspace(3) %ptr, i48 %arg) #0 {
define amdgpu_kernel void @local_store_i65(ptr addrspace(3) %ptr, i65 %arg) #0 {
; HAWAII-LABEL: local_store_i65:
; HAWAII: ; %bb.0:
-; HAWAII-NEXT: s_load_dword s2, s[8:9], 0x4
+; HAWAII-NEXT: s_load_dword s2, s[8:9], 0x6
; HAWAII-NEXT: s_load_dword s3, s[8:9], 0x0
-; HAWAII-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x2
+; HAWAII-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x4
; HAWAII-NEXT: s_mov_b32 m0, -1
; HAWAII-NEXT: s_waitcnt lgkmcnt(0)
; HAWAII-NEXT: s_and_b32 s2, s2, 1
@@ -268,9 +268,9 @@ define amdgpu_kernel void @local_store_i65(ptr addrspace(3) %ptr, i65 %arg) #0 {
;
; FIJI-LABEL: local_store_i65:
; FIJI: ; %bb.0:
-; FIJI-NEXT: s_load_dword s2, s[8:9], 0x10
+; FIJI-NEXT: s_load_dword s2, s[8:9], 0x18
; FIJI-NEXT: s_load_dword s3, s[8:9], 0x0
-; FIJI-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x8
+; FIJI-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x10
; FIJI-NEXT: s_mov_b32 m0, -1
; FIJI-NEXT: s_waitcnt lgkmcnt(0)
; FIJI-NEXT: s_and_b32 s2, s2, 1
@@ -284,9 +284,9 @@ define amdgpu_kernel void @local_store_i65(ptr addrspace(3) %ptr, i65 %arg) #0 {
;
; GFX9-LABEL: local_store_i65:
; GFX9: ; %bb.0:
-; GFX9-NEXT: s_load_dword s2, s[8:9], 0x10
+; GFX9-NEXT: s_load_dword s2, s[8:9], 0x18
; GFX9-NEXT: s_load_dword s3, s[8:9], 0x0
-; GFX9-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x8
+; GFX9-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x10
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: s_and_b32 s2, s2, 1
; GFX9-NEXT: v_mov_b32_e32 v2, s3
@@ -300,9 +300,9 @@ define amdgpu_kernel void @local_store_i65(ptr addrspace(3) %ptr, i65 %arg) #0 {
; GFX10-LABEL: local_store_i65:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_clause 0x2
-; GFX10-NEXT: s_load_dword s2, s[8:9], 0x10
+; GFX10-NEXT: s_load_dword s2, s[8:9], 0x18
; GFX10-NEXT: s_load_dword s3, s[8:9], 0x0
-; GFX10-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x8
+; GFX10-NEXT: s_load_dwordx2 s[0:1], s[8:9], 0x10
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
; GFX10-NEXT: s_and_b32 s2, s2, 1
; GFX10-NEXT: v_mov_b32_e32 v2, s3
@@ -316,9 +316,9 @@ define amdgpu_kernel void @local_store_i65(ptr addrspace(3) %ptr, i65 %arg) #0 {
; GFX11-LABEL: local_store_i65:
; GFX11: ; %bb.0:
; GFX11-NEXT: s_clause 0x2
-; GFX11-NEXT: s_load_b32 s2, s[4:5], 0x10
+; GFX11-NEXT: s_load_b32 s2, s[4:5], 0x18
; GFX11-NEXT: s_load_b32 s3, s[4:5], 0x0
-; GFX11-NEXT: s_load_b64 s[0:1], s[4:5], 0x8
+; GFX11-NEXT: s_load_b64 s[0:1], s[4:5], 0x10
; GFX11-NEXT: s_waitcnt lgkmcnt(0)
; GFX11-NEXT: s_and_b32 s2, s2, 1
; GFX11-NEXT: s_delay_alu instid0(SALU_CYCLE_1)
diff --git a/llvm/test/CodeGen/ARM/cmse-harden-entry-arguments.ll b/llvm/test/CodeGen/ARM/cmse-harden-entry-arguments.ll
index c66ab00566ddf..395557f2be13b 100644
--- a/llvm/test/CodeGen/ARM/cmse-harden-entry-arguments.ll
+++ b/llvm/test/CodeGen/ARM/cmse-harden-entry-arguments.ll
@@ -293,12 +293,23 @@ define i32 @access_i65(ptr byval(i65) %0) "cmse_nonsecure_entry" {
; V8M-COMMON-LABEL: access_i65:
; V8M-COMMON: @ %bb.0: @ %entry
; V8M-COMMON-NEXT: sub sp, #16
-; V8M-COMMON-NEXT: stm.w sp, {r0, r1, r2, r3}
-; V8M-LE-NEXT: ldrb.w r0, [sp, #8]
+; V8M-COMMON-NEXT: push {r4, r6, r7, lr}
+; V8M-COMMON-NEXT: add r7, sp, #8
+; V8M-COMMON-NEXT: mov r4, sp
+; V8M-COMMON-NEXT: bfc r4, #0, #4
+; V8M-COMMON-NEXT: mov sp, r4
+; V8M-LE-NEXT: add.w r12, r7, #8
+; V8M-LE-NEXT: sub.w r4, r7, #8
+; V8M-BE-NEXT: sub.w r4, r7, #8
+; V8M-BE-NEXT: add.w r12, r7, #8
+; V8M-COMMON-NEXT: stm.w r12, {r0, r1, r2, r3}
+; V8M-LE-NEXT: ldrb r0, [r7, #16]
; V8M-LE-NEXT: and r0, r0, #1
; V8M-LE-NEXT: rsbs r0, r0, #0
; V8M-BE-NEXT: movs r1, #0
; V8M-BE-NEXT: sub.w r0, r1, r0, lsr #24
+; V8M-COMMON-NEXT: mov sp, r4
+; V8M-COMMON-NEXT: pop.w {r4, r6, r7, lr}
; V8M-COMMON-NEXT: add sp, #16
; V8M-COMMON-NEXT: mov r1, lr
; V8M-COMMON-NEXT: mov r2, lr
@@ -311,14 +322,24 @@ define i32 @access_i65(ptr byval(i65) %0) "cmse_nonsecure_entry" {
; V81M-COMMON: @ %bb.0: @ %entry
; V81M-COMMON-NEXT: vstr fpcxtns, [sp, #-4]!
; V81M-COMMON-NEXT: sub sp, #16
+; V81M-COMMON-NEXT: push {r4, r6, r7, lr}
+; V81M-COMMON-NEXT: add r7, sp, #12
; V81M-COMMON-NEXT: add sp, #4
-; V81M-COMMON-NEXT: stm.w sp, {r0, r1, r2, r3}
-; V81M-LE-NEXT: ldrb.w r0, [sp, #8]
+; V81M-COMMON-NEXT: mov r4, sp
+; V81M-COMMON-NEXT: bfc r4, #0, #4
+; V81M-COMMON-NEXT: mov sp, r4
+; V81M-LE-NEXT: add.w r12, r7, #8
+; V81M-LE-NEXT: sub.w r4, r7, #12
+; V81M-BE-NEXT: sub.w r4, r7, #12
+; V81M-BE-NEXT: add.w r12, r7, #8
+; V81M-COMMON-NEXT: stm.w r12, {r0, r1, r2, r3}
+; V81M-LE-NEXT: ldrb r0, [r7, #16]
; V81M-LE-NEXT: and r0, r0, #1
; V81M-LE-NEXT: rsbs r0, r0, #0
; V81M-BE-NEXT: movs r1, #0
; V81M-BE-NEXT: sub.w r0, r1, r0, lsr #24
-; V81M-COMMON-NEXT: sub sp, #4
+; V81M-COMMON-NEXT: mov sp, r4
+; V81M-COMMON-NEXT: pop.w {r4, r6, r7, lr}
; V81M-COMMON-NEXT: add sp, #16
; V81M-COMMON-NEXT: vscclrm {s0, s1, s2, s3, s4, s5, s6, s7, s8, s9, s10, s11, s12, s13, s14, s15, vpr}
; V81M-COMMON-NEXT: vldr fpcxtns, [sp], #4
@@ -335,9 +356,20 @@ define i32 @access_u65(ptr byval(i65) %0) "cmse_nonsecure_entry" {
; V8M-COMMON-LABEL: access_u65:
; V8M-COMMON: @ %bb.0: @ %entry
; V8M-COMMON-NEXT: sub sp, #16
-; V8M-COMMON-NEXT: stm.w sp, {r0, r1, r2, r3}
-; V8M-LE-NEXT: ldrb.w r0, [sp, #8]
+; V8M-COMMON-NEXT: push {r4, r6, r7, lr}
+; V8M-COMMON-NEXT: add r7, sp, #8
+; V8M-COMMON-NEXT: mov r4, sp
+; V8M-COMMON-NEXT: bfc r4, #0, #4
+; V8M-COMMON-NEXT: mov sp, r4
+; V8M-LE-NEXT: add.w r12, r7, #8
+; V8M-LE-NEXT: sub.w r4, r7, #8
+; V8M-BE-NEXT: sub.w r4, r7, #8
+; V8M-BE-NEXT: add.w r12, r7, #8
+; V8M-COMMON-NEXT: stm.w r12, {r0, r1, r2, r3}
+; V8M-LE-NEXT: ldrb r0, [r7, #16]
; V8M-BE-NEXT: lsrs r0, r0, #24
+; V8M-COMMON-NEXT: mov sp, r4
+; V8M-COMMON-NEXT: pop.w {r4, r6, r7, lr}
; V8M-COMMON-NEXT: add sp, #16
; V8M-COMMON-NEXT: mov r1, lr
; V8M-COMMON-NEXT: mov r2, lr
@@ -350,11 +382,21 @@ define i32 @access_u65(ptr byval(i65) %0) "cmse_nonsecure_entry" {
; V81M-COMMON: @ %bb.0: @ %entry
; V81M-COMMON-NEXT: vstr fpcxtns, [sp, #-4]!
; V81M-COMMON-NEXT: sub sp, #16
+; V81M-COMMON-NEXT: push {r4, r6, r7, lr}
+; V81M-COMMON-NEXT: add r7, sp, #12
; V81M-COMMON-NEXT: add sp, #4
-; V81M-COMMON-NEXT: stm.w sp, {r0, r1, r2, r3}
-; V81M-LE-NEXT: ldrb.w r0, [sp, #8]
+; V81M-COMMON-NEXT: mov r4, sp
+; V81M-COMMON-NEXT: bfc r4, #0, #4
+; V81M-COMMON-NEXT: mov sp, r4
+; V81M-LE-NEXT: add.w r12, r7, #8
+; V81M-LE-NEXT: sub.w r4, r7, #12
+; V81M-BE-NEXT: sub.w r4, r7, #12
+; V81M-BE-NEXT: add.w r12, r7, #8
+; V81M-COMMON-NEXT: stm.w r12, {r0, r1, r2, r3}
+; V81M-LE-NEXT: ldrb r0, [r7, #16]
; V81M-BE-NEXT: lsrs r0, r0, #24
-; V81M-COMMON-NEXT: sub sp, #4
+; V81M-COMMON-NEXT: mov sp, r4
+; V81M-COMMON-NEXT: pop.w {r4, r6, r7, lr}
; V81M-COMMON-NEXT: add sp, #16
; V81M-COMMON-NEXT: vscclrm {s0, s1, s2, s3, s4, s5, s6, s7, s8, s9, s10, s11, s12, s13, s14, s15, vpr}
; V81M-COMMON-NEXT: vldr fpcxtns, [sp], #4
diff --git a/llvm/test/CodeGen/ARM/emit-big-cst.ll b/llvm/test/CodeGen/ARM/emit-big-cst.ll
index 0f451f3450fff..992a26373a6a8 100644
--- a/llvm/test/CodeGen/ARM/emit-big-cst.ll
+++ b/llvm/test/CodeGen/ARM/emit-big-cst.ll
@@ -6,8 +6,8 @@
; CHECK-NEXT: .long 2960197
; CHECK-NEXT: .short 26220
; CHECK-NEXT: .byte 0
-; CHECK-NEXT: .zero 1
-; CHECK-NEXT: .size bigCst, 12
+; CHECK-NEXT: .zero 5
+; CHECK-NEXT: .size bigCst, 16
@bigCst = internal constant i82 483673642326615442599424
diff --git a/llvm/test/CodeGen/ARM/llvm.sincos.ll b/llvm/test/CodeGen/ARM/llvm.sincos.ll
index 1448fac8d864f..2403d40951dcb 100644
--- a/llvm/test/CodeGen/ARM/llvm.sincos.ll
+++ b/llvm/test/CodeGen/ARM/llvm.sincos.ll
@@ -886,60 +886,68 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) {
; GNU-LABEL: test_sincos_f128:
; GNU: @ %bb.0:
; GNU-NEXT: push {r4, r5, r7, lr}
-; GNU-NEXT: sub sp, #40
+; GNU-NEXT: add r7, sp, #8
+; GNU-NEXT: sub sp, #48
+; GNU-NEXT: mov r4, sp
+; GNU-NEXT: bfc r4, #0, #4
+; GNU-NEXT: mov sp, r4
; GNU-NEXT: mov r12, r3
-; GNU-NEXT: ldr r3, [sp, #56]
-; GNU-NEXT: add.w lr, sp, #8
+; GNU-NEXT: ldr r3, [r7, #8]
+; GNU-NEXT: add.w lr, sp, #16
; GNU-NEXT: mov r4, r0
-; GNU-NEXT: add r0, sp, #24
+; GNU-NEXT: add r0, sp, #32
; GNU-NEXT: strd r0, lr, [sp]
; GNU-NEXT: mov r0, r1
; GNU-NEXT: mov r1, r2
; GNU-NEXT: mov r2, r12
; GNU-NEXT: bl sincosl
-; GNU-NEXT: ldrd r2, r3, [sp, #16]
-; GNU-NEXT: ldrd r12, r1, [sp, #8]
+; GNU-NEXT: ldrd r2, r3, [sp, #24]
+; GNU-NEXT: ldrd r12, r1, [sp, #16]
; GNU-NEXT: str r3, [r4, #28]
-; GNU-NEXT: ldrd r3, r5, [sp, #32]
-; GNU-NEXT: ldrd lr, r0, [sp, #24]
+; GNU-NEXT: ldrd r3, r5, [sp, #40]
+; GNU-NEXT: ldrd lr, r0, [sp, #32]
; GNU-NEXT: strd r1, r2, [r4, #20]
; GNU-NEXT: add.w r1, r4, #8
; GNU-NEXT: stm.w r1, {r3, r5, r12}
; GNU-NEXT: strd lr, r0, [r4]
-; GNU-NEXT: add sp, #40
+; GNU-NEXT: sub.w r4, r7, #8
+; GNU-NEXT: mov sp, r4
; GNU-NEXT: pop {r4, r5, r7, pc}
;
; GNUEABI-LABEL: test_sincos_f128:
; GNUEABI: @ %bb.0:
; GNUEABI-NEXT: .save {r4, r5, r11, lr}
; GNUEABI-NEXT: push {r4, r5, r11, lr}
-; GNUEABI-NEXT: .pad #40
-; GNUEABI-NEXT: sub sp, sp, #40
+; GNUEABI-NEXT: .setfp r11, sp, #8
+; GNUEABI-NEXT: add r11, sp, #8
+; GNUEABI-NEXT: .pad #48
+; GNUEABI-NEXT: sub sp, sp, #48
+; GNUEABI-NEXT: bfc sp, #0, #4
; GNUEABI-NEXT: mov r12, r3
-; GNUEABI-NEXT: ldr r3, [sp, #56]
+; GNUEABI-NEXT: ldr r3, [r11, #8]
; GNUEABI-NEXT: mov r4, r0
-; GNUEABI-NEXT: add r0, sp, #24
-; GNUEABI-NEXT: add r5, sp, #8
+; GNUEABI-NEXT: add r0, sp, #32
+; GNUEABI-NEXT: add r5, sp, #16
; GNUEABI-NEXT: stm sp, {r0, r5}
; GNUEABI-NEXT: mov r0, r1
; GNUEABI-NEXT: mov r1, r2
; GNUEABI-NEXT: mov r2, r12
; GNUEABI-NEXT: bl sincosl
-; GNUEABI-NEXT: add r3, sp, #12
-; GNUEABI-NEXT: ldr r12, [sp, #8]
+; GNUEABI-NEXT: add r3, sp, #20
+; GNUEABI-NEXT: ldr r12, [sp, #16]
; GNUEABI-NEXT: ldm r3, {r1, r2, r3}
; GNUEABI-NEXT: str r3, [r4, #28]
-; GNUEABI-NEXT: ldr r0, [sp, #32]
-; GNUEABI-NEXT: ldr lr, [sp, #24]
-; GNUEABI-NEXT: ldr r5, [sp, #28]
-; GNUEABI-NEXT: ldr r3, [sp, #36]
+; GNUEABI-NEXT: ldr r0, [sp, #40]
+; GNUEABI-NEXT: ldr lr, [sp, #32]
+; GNUEABI-NEXT: ldr r5, [sp, #36]
+; GNUEABI-NEXT: ldr r3, [sp, #44]
; GNUEABI-NEXT: str r2, [r4, #24]
; GNUEABI-NEXT: str r1, [r4, #20]
; GNUEABI-NEXT: add r1, r4, #8
; GNUEABI-NEXT: stm r1, {r0, r3, r12}
; GNUEABI-NEXT: str r5, [r4, #4]
; GNUEABI-NEXT: str lr, [r4]
-; GNUEABI-NEXT: add sp, sp, #40
+; GNUEABI-NEXT: sub sp, r11, #8
; GNUEABI-NEXT: pop {r4, r5, r11, pc}
;
; IOS-LABEL: test_sincos_f128:
diff --git a/llvm/test/CodeGen/ARM/store-postinc.ll b/llvm/test/CodeGen/ARM/store-postinc.ll
index 2735819b2a71f..193bd9ec85dfc 100644
--- a/llvm/test/CodeGen/ARM/store-postinc.ll
+++ b/llvm/test/CodeGen/ARM/store-postinc.ll
@@ -789,29 +789,23 @@ define ptr @i64_m4096(ptr %p, i64 %v) {
define ptr @i128_0(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_0:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #12]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #8]
-; CHECK-T1-NEXT: stm r0!, {r2, r3}
-; CHECK-T1-NEXT: subs r0, #8
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: stm r0!, {r1, r2, r3, r4}
+; CHECK-T1-NEXT: subs r0, #16
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_0:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
-; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: str r1, [r0, #8]
-; CHECK-T2-NEXT: strd r2, r3, [r0]
+; CHECK-T2-NEXT: ldr.w r12, [sp]
+; CHECK-T2-NEXT: stm.w r0, {r1, r2, r3, r12}
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_0:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: stm r0, {r2, r3}
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stm r0, {r1, r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
store i128 %v, ptr %p, align 16
ret ptr %p
@@ -820,32 +814,30 @@ define ptr @i128_0(ptr %p, i128 %v) {
define ptr @i128_3(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_3:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: movs r1, #3
-; CHECK-T1-NEXT: str r2, [r0, r1]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: movs r4, #3
+; CHECK-T1-NEXT: str r1, [r0, r4]
; CHECK-T1-NEXT: adds r0, r0, #3
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
+; CHECK-T1-NEXT: ldr r1, [sp, #8]
+; CHECK-T1-NEXT: str r2, [r0, #4]
+; CHECK-T1-NEXT: str r3, [r0, #8]
; CHECK-T1-NEXT: str r1, [r0, #12]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #8]
-; CHECK-T1-NEXT: str r3, [r0, #4]
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_3:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #3]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #3]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_3:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #3]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #3]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 3
store i128 %v, ptr %o, align 16
@@ -855,31 +847,29 @@ define ptr @i128_3(ptr %p, i128 %v) {
define ptr @i128_4(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_4:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #16]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #12]
-; CHECK-T1-NEXT: str r3, [r0, #8]
-; CHECK-T1-NEXT: str r2, [r0, #4]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: str r1, [r0, #4]
+; CHECK-T1-NEXT: str r2, [r0, #8]
+; CHECK-T1-NEXT: str r3, [r0, #12]
+; CHECK-T1-NEXT: str r4, [r0, #16]
; CHECK-T1-NEXT: adds r0, r0, #4
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_4:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #4]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #4]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_4:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #4]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #4]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 4
store i128 %v, ptr %o, align 16
@@ -889,31 +879,29 @@ define ptr @i128_4(ptr %p, i128 %v) {
define ptr @i128_8(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_8:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #20]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #16]
-; CHECK-T1-NEXT: str r3, [r0, #12]
-; CHECK-T1-NEXT: str r2, [r0, #8]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: str r1, [r0, #8]
+; CHECK-T1-NEXT: str r2, [r0, #12]
+; CHECK-T1-NEXT: str r3, [r0, #16]
+; CHECK-T1-NEXT: str r4, [r0, #20]
; CHECK-T1-NEXT: adds r0, #8
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_8:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #8]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #8]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_8:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #8]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #8]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 8
store i128 %v, ptr %o, align 16
@@ -923,31 +911,29 @@ define ptr @i128_8(ptr %p, i128 %v) {
define ptr @i128_16(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_16:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #28]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #24]
-; CHECK-T1-NEXT: str r3, [r0, #20]
-; CHECK-T1-NEXT: str r2, [r0, #16]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: str r1, [r0, #16]
+; CHECK-T1-NEXT: str r2, [r0, #20]
+; CHECK-T1-NEXT: str r3, [r0, #24]
+; CHECK-T1-NEXT: str r4, [r0, #28]
; CHECK-T1-NEXT: adds r0, #16
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_16:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #16]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #16]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_16:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #16]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #16]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 16
store i128 %v, ptr %o, align 16
@@ -957,31 +943,27 @@ define ptr @i128_16(ptr %p, i128 %v) {
define ptr @i128_m1(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_m1:
; CHECK-T1: @ %bb.0:
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
; CHECK-T1-NEXT: subs r0, r0, #1
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #12]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #8]
-; CHECK-T1-NEXT: stm r0!, {r2, r3}
-; CHECK-T1-NEXT: subs r0, #8
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: stm r0!, {r1, r2, r3, r4}
+; CHECK-T1-NEXT: subs r0, #16
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_m1:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #-1]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #-1]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_m1:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #-1]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #-1]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 -1
store i128 %v, ptr %o, align 16
@@ -991,30 +973,27 @@ define ptr @i128_m1(ptr %p, i128 %v) {
define ptr @i128_m4(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_m4:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #8]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #4]
-; CHECK-T1-NEXT: str r3, [r0]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: stm r0!, {r2, r3, r4}
+; CHECK-T1-NEXT: subs r0, #12
; CHECK-T1-NEXT: subs r0, r0, #4
-; CHECK-T1-NEXT: str r2, [r0]
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: str r1, [r0]
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_m4:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #8]
-; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0]
-; CHECK-T2-NEXT: str r2, [r0, #-4]!
+; CHECK-T2-NEXT: ldr.w r12, [sp]
+; CHECK-T2-NEXT: stm.w r0, {r2, r3, r12}
+; CHECK-T2-NEXT: str r1, [r0, #-4]!
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_m4:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0]
-; CHECK-ARM-NEXT: stmib r0, {r1, r12}
-; CHECK-ARM-NEXT: str r2, [r0, #-4]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stm r0, {r2, r3, r12}
+; CHECK-ARM-NEXT: str r1, [r0, #-4]!
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 -4
store i128 %v, ptr %o, align 16
diff --git a/llvm/test/CodeGen/ARM/store-preinc.ll b/llvm/test/CodeGen/ARM/store-preinc.ll
index d3bf001ca307b..409bbdebff5b5 100644
--- a/llvm/test/CodeGen/ARM/store-preinc.ll
+++ b/llvm/test/CodeGen/ARM/store-preinc.ll
@@ -789,29 +789,23 @@ define ptr @i64_m4096(ptr %p, i64 %v) {
define ptr @i128_0(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_0:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #12]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #8]
-; CHECK-T1-NEXT: stm r0!, {r2, r3}
-; CHECK-T1-NEXT: subs r0, #8
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: stm r0!, {r1, r2, r3, r4}
+; CHECK-T1-NEXT: subs r0, #16
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_0:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
-; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: str r1, [r0, #8]
-; CHECK-T2-NEXT: strd r2, r3, [r0]
+; CHECK-T2-NEXT: ldr.w r12, [sp]
+; CHECK-T2-NEXT: stm.w r0, {r1, r2, r3, r12}
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_0:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: stm r0, {r2, r3}
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stm r0, {r1, r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
store i128 %v, ptr %p, align 16
ret ptr %p
@@ -820,32 +814,30 @@ define ptr @i128_0(ptr %p, i128 %v) {
define ptr @i128_3(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_3:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: movs r1, #3
-; CHECK-T1-NEXT: str r2, [r0, r1]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: movs r4, #3
+; CHECK-T1-NEXT: str r1, [r0, r4]
; CHECK-T1-NEXT: adds r0, r0, #3
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
+; CHECK-T1-NEXT: ldr r1, [sp, #8]
+; CHECK-T1-NEXT: str r2, [r0, #4]
+; CHECK-T1-NEXT: str r3, [r0, #8]
; CHECK-T1-NEXT: str r1, [r0, #12]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #8]
-; CHECK-T1-NEXT: str r3, [r0, #4]
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_3:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #3]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #3]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_3:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #3]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #3]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 3
store i128 %v, ptr %o, align 16
@@ -855,31 +847,29 @@ define ptr @i128_3(ptr %p, i128 %v) {
define ptr @i128_4(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_4:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #16]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #12]
-; CHECK-T1-NEXT: str r3, [r0, #8]
-; CHECK-T1-NEXT: str r2, [r0, #4]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: str r1, [r0, #4]
+; CHECK-T1-NEXT: str r2, [r0, #8]
+; CHECK-T1-NEXT: str r3, [r0, #12]
+; CHECK-T1-NEXT: str r4, [r0, #16]
; CHECK-T1-NEXT: adds r0, r0, #4
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_4:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #4]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #4]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_4:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #4]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #4]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 4
store i128 %v, ptr %o, align 16
@@ -889,31 +879,29 @@ define ptr @i128_4(ptr %p, i128 %v) {
define ptr @i128_8(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_8:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #20]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #16]
-; CHECK-T1-NEXT: str r3, [r0, #12]
-; CHECK-T1-NEXT: str r2, [r0, #8]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: str r1, [r0, #8]
+; CHECK-T1-NEXT: str r2, [r0, #12]
+; CHECK-T1-NEXT: str r3, [r0, #16]
+; CHECK-T1-NEXT: str r4, [r0, #20]
; CHECK-T1-NEXT: adds r0, #8
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_8:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #8]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #8]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_8:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #8]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #8]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 8
store i128 %v, ptr %o, align 16
@@ -923,31 +911,29 @@ define ptr @i128_8(ptr %p, i128 %v) {
define ptr @i128_16(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_16:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #28]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #24]
-; CHECK-T1-NEXT: str r3, [r0, #20]
-; CHECK-T1-NEXT: str r2, [r0, #16]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: str r1, [r0, #16]
+; CHECK-T1-NEXT: str r2, [r0, #20]
+; CHECK-T1-NEXT: str r3, [r0, #24]
+; CHECK-T1-NEXT: str r4, [r0, #28]
; CHECK-T1-NEXT: adds r0, #16
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_16:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #16]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #16]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_16:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #16]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #16]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 16
store i128 %v, ptr %o, align 16
@@ -957,31 +943,27 @@ define ptr @i128_16(ptr %p, i128 %v) {
define ptr @i128_m1(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_m1:
; CHECK-T1: @ %bb.0:
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
; CHECK-T1-NEXT: subs r0, r0, #1
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #12]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #8]
-; CHECK-T1-NEXT: stm r0!, {r2, r3}
-; CHECK-T1-NEXT: subs r0, #8
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: stm r0!, {r1, r2, r3, r4}
+; CHECK-T1-NEXT: subs r0, #16
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_m1:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: str r2, [r0, #-1]!
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #12]
+; CHECK-T2-NEXT: str r1, [r0, #-1]!
; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0, #4]
+; CHECK-T2-NEXT: strd r2, r3, [r0, #4]
+; CHECK-T2-NEXT: str r1, [r0, #12]
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_m1:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: str r2, [r0, #-1]!
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0, #4]
-; CHECK-ARM-NEXT: str r1, [r0, #8]
-; CHECK-ARM-NEXT: str r12, [r0, #12]
+; CHECK-ARM-NEXT: str r1, [r0, #-1]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stmib r0, {r2, r3, r12}
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 -1
store i128 %v, ptr %o, align 16
@@ -991,30 +973,27 @@ define ptr @i128_m1(ptr %p, i128 %v) {
define ptr @i128_m4(ptr %p, i128 %v) {
; CHECK-T1-LABEL: i128_m4:
; CHECK-T1: @ %bb.0:
-; CHECK-T1-NEXT: ldr r1, [sp, #4]
-; CHECK-T1-NEXT: str r1, [r0, #8]
-; CHECK-T1-NEXT: ldr r1, [sp]
-; CHECK-T1-NEXT: str r1, [r0, #4]
-; CHECK-T1-NEXT: str r3, [r0]
+; CHECK-T1-NEXT: .save {r4, lr}
+; CHECK-T1-NEXT: push {r4, lr}
+; CHECK-T1-NEXT: ldr r4, [sp, #8]
+; CHECK-T1-NEXT: stm r0!, {r2, r3, r4}
+; CHECK-T1-NEXT: subs r0, #12
; CHECK-T1-NEXT: subs r0, r0, #4
-; CHECK-T1-NEXT: str r2, [r0]
-; CHECK-T1-NEXT: bx lr
+; CHECK-T1-NEXT: str r1, [r0]
+; CHECK-T1-NEXT: pop {r4, pc}
;
; CHECK-T2-LABEL: i128_m4:
; CHECK-T2: @ %bb.0:
-; CHECK-T2-NEXT: ldr r1, [sp, #4]
-; CHECK-T2-NEXT: str r1, [r0, #8]
-; CHECK-T2-NEXT: ldr r1, [sp]
-; CHECK-T2-NEXT: strd r3, r1, [r0]
-; CHECK-T2-NEXT: str r2, [r0, #-4]!
+; CHECK-T2-NEXT: ldr.w r12, [sp]
+; CHECK-T2-NEXT: stm.w r0, {r2, r3, r12}
+; CHECK-T2-NEXT: str r1, [r0, #-4]!
; CHECK-T2-NEXT: bx lr
;
; CHECK-ARM-LABEL: i128_m4:
; CHECK-ARM: @ %bb.0:
-; CHECK-ARM-NEXT: ldm sp, {r1, r12}
-; CHECK-ARM-NEXT: str r3, [r0]
-; CHECK-ARM-NEXT: stmib r0, {r1, r12}
-; CHECK-ARM-NEXT: str r2, [r0, #-4]!
+; CHECK-ARM-NEXT: ldr r12, [sp]
+; CHECK-ARM-NEXT: stm r0, {r2, r3, r12}
+; CHECK-ARM-NEXT: str r1, [r0, #-4]!
; CHECK-ARM-NEXT: bx lr
%o = getelementptr inbounds i8, ptr %p, i32 -4
store i128 %v, ptr %o, align 16
diff --git a/llvm/test/CodeGen/ARM/umulo-128-legalisation-lowering.ll b/llvm/test/CodeGen/ARM/umulo-128-legalisation-lowering.ll
index 4eb82c80e2bff..aef18bea2769f 100644
--- a/llvm/test/CodeGen/ARM/umulo-128-legalisation-lowering.ll
+++ b/llvm/test/CodeGen/ARM/umulo-128-legalisation-lowering.ll
@@ -7,96 +7,98 @@ define { i128, i8 } @muloti_test(i128 %l, i128 %r) unnamed_addr #0 {
; ARMV6: @ %bb.0: @ %start
; ARMV6-NEXT: push {r4, r5, r6, r7, r8, r9, r10, r11, lr}
; ARMV6-NEXT: sub sp, sp, #28
+; ARMV6-NEXT: ldr r5, [sp, #68]
+; ARMV6-NEXT: mov r6, r0
+; ARMV6-NEXT: str r0, [sp] @ 4-byte Spill
; ARMV6-NEXT: ldr r4, [sp, #72]
-; ARMV6-NEXT: mov r7, r0
-; ARMV6-NEXT: str r0, [sp, #4] @ 4-byte Spill
-; ARMV6-NEXT: ldr r12, [sp, #64]
-; ARMV6-NEXT: umull r1, r0, r2, r4
+; ARMV6-NEXT: umull r7, r0, r1, r5
+; ARMV6-NEXT: str r7, [r6]
+; ARMV6-NEXT: umull r12, r7, r4, r3
+; ARMV6-NEXT: ldr r6, [sp, #64]
+; ARMV6-NEXT: str r3, [sp, #8] @ 4-byte Spill
+; ARMV6-NEXT: str r7, [sp, #4] @ 4-byte Spill
+; ARMV6-NEXT: umull r7, r9, r6, r5
+; ARMV6-NEXT: add r12, r7, r12
+; ARMV6-NEXT: umull r3, r7, r3, r5
+; ARMV6-NEXT: ldr r5, [sp, #76]
+; ARMV6-NEXT: str r3, [sp, #16] @ 4-byte Spill
+; ARMV6-NEXT: mov r3, #0
+; ARMV6-NEXT: adds lr, r7, r12
+; ARMV6-NEXT: ldr r7, [sp, #80]
+; ARMV6-NEXT: adc r3, r3, #0
+; ARMV6-NEXT: str r3, [sp, #24] @ 4-byte Spill
+; ARMV6-NEXT: umull r3, r10, r2, r5
+; ARMV6-NEXT: umull r11, r12, r7, r1
+; ARMV6-NEXT: add r3, r11, r3
+; ARMV6-NEXT: umull r11, r8, r5, r1
+; ARMV6-NEXT: mov r5, #0
+; ARMV6-NEXT: adds r3, r8, r3
+; ARMV6-NEXT: adc r8, r5, #0
+; ARMV6-NEXT: ldr r5, [sp, #16] @ 4-byte Reload
+; ARMV6-NEXT: str r8, [sp, #20] @ 4-byte Spill
+; ARMV6-NEXT: adds r5, r5, r11
+; ARMV6-NEXT: str r5, [sp, #12] @ 4-byte Spill
; ARMV6-NEXT: ldr r5, [sp, #68]
-; ARMV6-NEXT: str r1, [r7]
+; ARMV6-NEXT: adc r3, lr, r3
+; ARMV6-NEXT: str r3, [sp, #16] @ 4-byte Spill
+; ARMV6-NEXT: mov r3, #0
+; ARMV6-NEXT: umull r1, lr, r1, r4
+; ARMV6-NEXT: umlal r0, r3, r2, r5
+; ARMV6-NEXT: ldr r5, [sp] @ 4-byte Reload
+; ARMV6-NEXT: adds r0, r1, r0
+; ARMV6-NEXT: str r0, [r5, #4]
+; ARMV6-NEXT: adcs r0, r3, lr
+; ARMV6-NEXT: mov r1, #0
+; ARMV6-NEXT: adc lr, r1, #0
+; ARMV6-NEXT: cmp r10, #0
+; ARMV6-NEXT: ldr r1, [sp, #4] @ 4-byte Reload
+; ARMV6-NEXT: movne r10, #1
+; ARMV6-NEXT: cmp r12, #0
+; ARMV6-NEXT: umlal r0, lr, r2, r4
+; ARMV6-NEXT: movne r12, #1
+; ARMV6-NEXT: cmp r1, #0
+; ARMV6-NEXT: movne r1, #1
+; ARMV6-NEXT: cmp r9, #0
+; ARMV6-NEXT: mov r11, r1
+; ARMV6-NEXT: ldr r1, [sp, #8] @ 4-byte Reload
+; ARMV6-NEXT: movne r9, #1
+; ARMV6-NEXT: ldr r8, [sp, #12] @ 4-byte Reload
+; ARMV6-NEXT: orrs r3, r1, r6
; ARMV6-NEXT: ldr r1, [sp, #76]
-; ARMV6-NEXT: umull r7, r6, r1, r12
-; ARMV6-NEXT: str r6, [sp, #8] @ 4-byte Spill
-; ARMV6-NEXT: umull r6, r9, r5, r4
-; ARMV6-NEXT: add r7, r6, r7
-; ARMV6-NEXT: umull r4, r6, r12, r4
-; ARMV6-NEXT: str r4, [sp, #16] @ 4-byte Spill
-; ARMV6-NEXT: mov r4, #0
-; ARMV6-NEXT: adds r8, r6, r7
-; ARMV6-NEXT: ldr r6, [sp, #80]
-; ARMV6-NEXT: adc r7, r4, #0
-; ARMV6-NEXT: ldr r4, [sp, #84]
-; ARMV6-NEXT: str r7, [sp, #24] @ 4-byte Spill
-; ARMV6-NEXT: umull r12, lr, r3, r6
-; ARMV6-NEXT: umull r11, r7, r4, r2
-; ARMV6-NEXT: add r12, r11, r12
-; ARMV6-NEXT: umull r11, r10, r6, r2
-; ARMV6-NEXT: adds r12, r10, r12
-; ARMV6-NEXT: mov r10, #0
-; ARMV6-NEXT: adc r6, r10, #0
-; ARMV6-NEXT: str r6, [sp, #20] @ 4-byte Spill
-; ARMV6-NEXT: ldr r6, [sp, #16] @ 4-byte Reload
-; ARMV6-NEXT: adds r6, r6, r11
-; ARMV6-NEXT: str r6, [sp, #12] @ 4-byte Spill
-; ARMV6-NEXT: adc r6, r8, r12
-; ARMV6-NEXT: str r6, [sp, #16] @ 4-byte Spill
-; ARMV6-NEXT: ldr r6, [sp, #72]
-; ARMV6-NEXT: mov r12, #0
-; ARMV6-NEXT: umull r2, r8, r2, r1
-; ARMV6-NEXT: umlal r0, r12, r3, r6
-; ARMV6-NEXT: adds r0, r2, r0
-; ARMV6-NEXT: ldr r2, [sp, #4] @ 4-byte Reload
-; ARMV6-NEXT: adcs r8, r12, r8
-; ARMV6-NEXT: adc r12, r10, #0
-; ARMV6-NEXT: cmp lr, #0
-; ARMV6-NEXT: str r0, [r2, #4]
-; ARMV6-NEXT: movne lr, #1
-; ARMV6-NEXT: ldr r11, [sp, #8] @ 4-byte Reload
+; ARMV6-NEXT: movne r3, #1
+; ARMV6-NEXT: orrs r1, r1, r7
+; ARMV6-NEXT: movne r1, #1
; ARMV6-NEXT: cmp r7, #0
; ARMV6-NEXT: movne r7, #1
-; ARMV6-NEXT: ldr r0, [sp, #64]
-; ARMV6-NEXT: cmp r11, #0
-; ARMV6-NEXT: umlal r8, r12, r3, r1
-; ARMV6-NEXT: movne r11, #1
-; ARMV6-NEXT: cmp r9, #0
-; ARMV6-NEXT: movne r9, #1
-; ARMV6-NEXT: orrs r10, r0, r5
-; ARMV6-NEXT: ldr r0, [sp, #80]
-; ARMV6-NEXT: movne r10, #1
-; ARMV6-NEXT: ldr r6, [sp, #12] @ 4-byte Reload
-; ARMV6-NEXT: orrs r0, r0, r4
-; ARMV6-NEXT: movne r0, #1
+; ARMV6-NEXT: cmp r2, #0
+; ARMV6-NEXT: movne r2, #1
+; ARMV6-NEXT: cmp r6, #0
+; ARMV6-NEXT: movne r6, #1
; ARMV6-NEXT: cmp r4, #0
; ARMV6-NEXT: movne r4, #1
-; ARMV6-NEXT: cmp r3, #0
-; ARMV6-NEXT: movne r3, #1
-; ARMV6-NEXT: cmp r5, #0
-; ARMV6-NEXT: movne r5, #1
-; ARMV6-NEXT: cmp r1, #0
-; ARMV6-NEXT: movne r1, #1
-; ARMV6-NEXT: adds r6, r8, r6
-; ARMV6-NEXT: str r6, [r2, #8]
-; ARMV6-NEXT: and r1, r5, r1
-; ARMV6-NEXT: ldr r6, [sp, #16] @ 4-byte Reload
+; ARMV6-NEXT: adds r0, r0, r8
+; ARMV6-NEXT: str r0, [r5, #8]
+; ARMV6-NEXT: ldr r0, [sp, #16] @ 4-byte Reload
+; ARMV6-NEXT: adcs r0, lr, r0
+; ARMV6-NEXT: str r0, [r5, #12]
+; ARMV6-NEXT: and r0, r3, r1
+; ARMV6-NEXT: and r1, r6, r4
; ARMV6-NEXT: orr r1, r1, r9
+; ARMV6-NEXT: ldr r3, [sp, #24] @ 4-byte Reload
; ARMV6-NEXT: orr r1, r1, r11
-; ARMV6-NEXT: and r0, r10, r0
-; ARMV6-NEXT: adcs r6, r12, r6
-; ARMV6-NEXT: str r6, [r2, #12]
-; ARMV6-NEXT: ldr r6, [sp, #24] @ 4-byte Reload
-; ARMV6-NEXT: orr r1, r1, r6
-; ARMV6-NEXT: orr r0, r0, r1
-; ARMV6-NEXT: and r1, r4, r3
-; ARMV6-NEXT: orr r1, r1, r7
-; ARMV6-NEXT: ldr r3, [sp, #20] @ 4-byte Reload
-; ARMV6-NEXT: orr r1, r1, lr
; ARMV6-NEXT: orr r1, r1, r3
; ARMV6-NEXT: orr r0, r0, r1
+; ARMV6-NEXT: and r1, r7, r2
+; ARMV6-NEXT: orr r1, r1, r12
+; ARMV6-NEXT: ldr r2, [sp, #20] @ 4-byte Reload
+; ARMV6-NEXT: orr r1, r1, r10
+; ARMV6-NEXT: orr r1, r1, r2
+; ARMV6-NEXT: orr r0, r0, r1
; ARMV6-NEXT: mov r1, #0
; ARMV6-NEXT: adc r1, r1, #0
; ARMV6-NEXT: orr r0, r0, r1
; ARMV6-NEXT: and r0, r0, #1
-; ARMV6-NEXT: strb r0, [r2, #16]
+; ARMV6-NEXT: strb r0, [r5, #16]
; ARMV6-NEXT: add sp, sp, #28
; ARMV6-NEXT: pop {r4, r5, r6, r7, r8, r9, r10, r11, pc}
;
@@ -104,111 +106,109 @@ define { i128, i8 } @muloti_test(i128 %l, i128 %r) unnamed_addr #0 {
; ARMV7: @ %bb.0: @ %start
; ARMV7-NEXT: push {r4, r5, r6, r7, r8, r9, r10, r11, lr}
; ARMV7-NEXT: sub sp, sp, #44
-; ARMV7-NEXT: ldr r8, [sp, #88]
-; ARMV7-NEXT: mov r9, r0
-; ARMV7-NEXT: ldr r7, [sp, #96]
-; ARMV7-NEXT: ldr lr, [sp, #100]
-; ARMV7-NEXT: umull r0, r5, r2, r8
-; ARMV7-NEXT: ldr r4, [sp, #80]
+; ARMV7-NEXT: ldr r6, [sp, #84]
+; ARMV7-NEXT: mov r8, r0
+; ARMV7-NEXT: ldr r7, [sp, #92]
+; ARMV7-NEXT: mov lr, r3
+; ARMV7-NEXT: ldr r4, [sp, #96]
+; ARMV7-NEXT: umull r0, r5, r1, r6
+; ARMV7-NEXT: ldr r9, [sp, #88]
; ARMV7-NEXT: str r0, [sp, #32] @ 4-byte Spill
-; ARMV7-NEXT: umull r1, r0, r3, r7
+; ARMV7-NEXT: umull r3, r0, r2, r7
; ARMV7-NEXT: str r0, [sp, #4] @ 4-byte Spill
-; ARMV7-NEXT: umull r0, r11, lr, r2
-; ARMV7-NEXT: str r1, [sp, #20] @ 4-byte Spill
-; ARMV7-NEXT: ldr r1, [sp, #92]
+; ARMV7-NEXT: umull r0, r10, r4, r1
+; ARMV7-NEXT: str r3, [sp, #20] @ 4-byte Spill
; ARMV7-NEXT: str r0, [sp] @ 4-byte Spill
-; ARMV7-NEXT: umull r0, r10, r7, r2
-; ARMV7-NEXT: mov r7, r1
-; ARMV7-NEXT: umull r6, r12, r1, r4
+; ARMV7-NEXT: umull r0, r11, r7, r1
+; ARMV7-NEXT: umull r7, r12, r9, lr
; ARMV7-NEXT: str r0, [sp, #40] @ 4-byte Spill
-; ARMV7-NEXT: ldr r0, [sp, #84]
-; ARMV7-NEXT: str r6, [sp, #24] @ 4-byte Spill
-; ARMV7-NEXT: umull r6, r1, r0, r8
-; ARMV7-NEXT: str r6, [sp, #16] @ 4-byte Spill
-; ARMV7-NEXT: umull r6, r2, r2, r7
-; ARMV7-NEXT: mov r7, r4
-; ARMV7-NEXT: str r6, [sp, #8] @ 4-byte Spill
-; ARMV7-NEXT: str r2, [sp, #12] @ 4-byte Spill
-; ARMV7-NEXT: umull r2, r6, r4, r8
-; ARMV7-NEXT: str r2, [sp, #36] @ 4-byte Spill
-; ARMV7-NEXT: ldr r2, [sp, #32] @ 4-byte Reload
-; ARMV7-NEXT: str r6, [sp, #28] @ 4-byte Spill
-; ARMV7-NEXT: mov r6, #0
-; ARMV7-NEXT: str r2, [r9]
-; ARMV7-NEXT: umlal r5, r6, r3, r8
-; ARMV7-NEXT: ldr r2, [sp, #20] @ 4-byte Reload
-; ARMV7-NEXT: ldr r4, [sp] @ 4-byte Reload
-; ARMV7-NEXT: add r4, r4, r2
-; ARMV7-NEXT: adds r2, r10, r4
-; ARMV7-NEXT: str r2, [sp, #20] @ 4-byte Spill
-; ARMV7-NEXT: mov r2, #0
-; ARMV7-NEXT: adc r2, r2, #0
+; ARMV7-NEXT: ldr r0, [sp, #80]
+; ARMV7-NEXT: str r7, [sp, #24] @ 4-byte Spill
+; ARMV7-NEXT: umull r7, r3, r0, r6
+; ARMV7-NEXT: str r7, [sp, #16] @ 4-byte Spill
+; ARMV7-NEXT: umull r7, r1, r1, r9
+; ARMV7-NEXT: mov r9, #0
+; ARMV7-NEXT: str r7, [sp, #8] @ 4-byte Spill
+; ARMV7-NEXT: str r1, [sp, #12] @ 4-byte Spill
+; ARMV7-NEXT: umull r1, r7, lr, r6
+; ARMV7-NEXT: str r1, [sp, #36] @ 4-byte Spill
+; ARMV7-NEXT: ldr r1, [sp, #32] @ 4-byte Reload
+; ARMV7-NEXT: str r7, [sp, #28] @ 4-byte Spill
+; ARMV7-NEXT: mov r7, #0
+; ARMV7-NEXT: str r1, [r8]
+; ARMV7-NEXT: umlal r5, r7, r2, r6
+; ARMV7-NEXT: ldr r1, [sp, #20] @ 4-byte Reload
+; ARMV7-NEXT: ldr r6, [sp] @ 4-byte Reload
+; ARMV7-NEXT: add r6, r6, r1
+; ARMV7-NEXT: adds r1, r11, r6
+; ARMV7-NEXT: str r1, [sp, #20] @ 4-byte Spill
+; ARMV7-NEXT: adc r1, r9, #0
; ARMV7-NEXT: cmp r12, #0
-; ARMV7-NEXT: str r2, [sp, #32] @ 4-byte Spill
+; ARMV7-NEXT: str r1, [sp, #32] @ 4-byte Spill
; ARMV7-NEXT: movwne r12, #1
-; ARMV7-NEXT: cmp r1, #0
-; ARMV7-NEXT: ldr r2, [sp, #96]
-; ARMV7-NEXT: movwne r1, #1
-; ARMV7-NEXT: orrs r10, r7, r0
-; ARMV7-NEXT: movwne r10, #1
-; ARMV7-NEXT: orrs r7, r2, lr
-; ARMV7-NEXT: ldr r2, [sp, #92]
-; ARMV7-NEXT: movwne r7, #1
+; ARMV7-NEXT: cmp r3, #0
+; ARMV7-NEXT: ldr r1, [sp, #92]
+; ARMV7-NEXT: movwne r3, #1
+; ARMV7-NEXT: orrs lr, lr, r0
+; ARMV7-NEXT: movwne lr, #1
+; ARMV7-NEXT: orrs r9, r1, r4
+; ARMV7-NEXT: ldr r1, [sp, #88]
+; ARMV7-NEXT: movwne r9, #1
; ARMV7-NEXT: cmp r0, #0
+; ARMV7-NEXT: and lr, lr, r9
; ARMV7-NEXT: movwne r0, #1
-; ARMV7-NEXT: cmp r2, #0
-; ARMV7-NEXT: mov r4, r2
-; ARMV7-NEXT: mov r8, r2
-; ARMV7-NEXT: ldr r2, [sp, #8] @ 4-byte Reload
-; ARMV7-NEXT: movwne r4, #1
-; ARMV7-NEXT: and r0, r0, r4
-; ARMV7-NEXT: mov r4, #0
-; ARMV7-NEXT: adds r5, r2, r5
-; ARMV7-NEXT: str r5, [r9, #4]
-; ARMV7-NEXT: orr r0, r0, r1
+; ARMV7-NEXT: cmp r1, #0
+; ARMV7-NEXT: mov r6, r1
+; ARMV7-NEXT: mov r11, r1
+; ARMV7-NEXT: ldr r1, [sp, #8] @ 4-byte Reload
+; ARMV7-NEXT: movwne r6, #1
+; ARMV7-NEXT: and r0, r0, r6
+; ARMV7-NEXT: adds r5, r1, r5
+; ARMV7-NEXT: str r5, [r8, #4]
+; ARMV7-NEXT: orr r0, r0, r3
; ARMV7-NEXT: ldr r1, [sp, #24] @ 4-byte Reload
-; ARMV7-NEXT: ldr r2, [sp, #16] @ 4-byte Reload
-; ARMV7-NEXT: and r5, r10, r7
+; ARMV7-NEXT: ldr r3, [sp, #16] @ 4-byte Reload
+; ARMV7-NEXT: mov r5, #0
; ARMV7-NEXT: orr r0, r0, r12
; ARMV7-NEXT: mov r12, #0
-; ARMV7-NEXT: add r1, r2, r1
-; ARMV7-NEXT: ldr r2, [sp, #12] @ 4-byte Reload
-; ARMV7-NEXT: adcs r2, r6, r2
-; ARMV7-NEXT: ldr r6, [sp, #28] @ 4-byte Reload
-; ARMV7-NEXT: adc r7, r4, #0
-; ARMV7-NEXT: adds r1, r6, r1
-; ARMV7-NEXT: umlal r2, r7, r3, r8
-; ARMV7-NEXT: adc r4, r4, #0
-; ARMV7-NEXT: orr r0, r0, r4
-; ARMV7-NEXT: orr r0, r5, r0
-; ARMV7-NEXT: ldr r4, [sp, #40] @ 4-byte Reload
-; ARMV7-NEXT: ldr r5, [sp, #36] @ 4-byte Reload
-; ARMV7-NEXT: adds r5, r5, r4
-; ARMV7-NEXT: ldr r4, [sp, #20] @ 4-byte Reload
-; ARMV7-NEXT: adc r1, r1, r4
-; ARMV7-NEXT: ldr r4, [sp, #4] @ 4-byte Reload
+; ARMV7-NEXT: add r3, r3, r1
+; ARMV7-NEXT: ldr r1, [sp, #12] @ 4-byte Reload
+; ARMV7-NEXT: adcs r1, r7, r1
+; ARMV7-NEXT: ldr r7, [sp, #28] @ 4-byte Reload
+; ARMV7-NEXT: adc r6, r5, #0
+; ARMV7-NEXT: adds r3, r7, r3
+; ARMV7-NEXT: umlal r1, r6, r2, r11
+; ARMV7-NEXT: adc r5, r5, #0
+; ARMV7-NEXT: orr r0, r0, r5
+; ARMV7-NEXT: orr lr, lr, r0
+; ARMV7-NEXT: ldr r5, [sp, #40] @ 4-byte Reload
+; ARMV7-NEXT: ldr r0, [sp, #36] @ 4-byte Reload
+; ARMV7-NEXT: adds r5, r0, r5
+; ARMV7-NEXT: ldr r0, [sp, #20] @ 4-byte Reload
+; ARMV7-NEXT: adc r3, r3, r0
+; ARMV7-NEXT: ldr r0, [sp, #4] @ 4-byte Reload
+; ARMV7-NEXT: cmp r0, #0
+; ARMV7-NEXT: movwne r0, #1
+; ARMV7-NEXT: cmp r2, #0
+; ARMV7-NEXT: movwne r2, #1
; ARMV7-NEXT: cmp r4, #0
; ARMV7-NEXT: movwne r4, #1
-; ARMV7-NEXT: cmp r3, #0
-; ARMV7-NEXT: movwne r3, #1
-; ARMV7-NEXT: cmp lr, #0
-; ARMV7-NEXT: movwne lr, #1
-; ARMV7-NEXT: cmp r11, #0
-; ARMV7-NEXT: movwne r11, #1
-; ARMV7-NEXT: adds r2, r2, r5
-; ARMV7-NEXT: and r3, lr, r3
-; ARMV7-NEXT: str r2, [r9, #8]
-; ARMV7-NEXT: adcs r1, r7, r1
-; ARMV7-NEXT: str r1, [r9, #12]
-; ARMV7-NEXT: orr r1, r3, r11
-; ARMV7-NEXT: ldr r2, [sp, #32] @ 4-byte Reload
-; ARMV7-NEXT: orr r1, r1, r4
-; ARMV7-NEXT: orr r1, r1, r2
-; ARMV7-NEXT: orr r0, r0, r1
+; ARMV7-NEXT: cmp r10, #0
+; ARMV7-NEXT: movwne r10, #1
+; ARMV7-NEXT: adds r1, r1, r5
+; ARMV7-NEXT: str r1, [r8, #8]
+; ARMV7-NEXT: adcs r1, r6, r3
+; ARMV7-NEXT: and r2, r4, r2
+; ARMV7-NEXT: str r1, [r8, #12]
+; ARMV7-NEXT: orr r1, r2, r10
+; ARMV7-NEXT: orr r1, r1, r0
+; ARMV7-NEXT: ldr r0, [sp, #32] @ 4-byte Reload
+; ARMV7-NEXT: orr r1, r1, r0
+; ARMV7-NEXT: orr r0, lr, r1
; ARMV7-NEXT: adc r1, r12, #0
; ARMV7-NEXT: orr r0, r0, r1
; ARMV7-NEXT: and r0, r0, #1
-; ARMV7-NEXT: strb r0, [r9, #16]
+; ARMV7-NEXT: strb r0, [r8, #16]
; ARMV7-NEXT: add sp, sp, #44
; ARMV7-NEXT: pop {r4, r5, r6, r7, r8, r9, r10, r11, pc}
start:
diff --git a/llvm/test/CodeGen/ARM/unaligned_load_store_aeabi.ll b/llvm/test/CodeGen/ARM/unaligned_load_store_aeabi.ll
index 5e395c49cd3b2..aca0eba7457a9 100644
--- a/llvm/test/CodeGen/ARM/unaligned_load_store_aeabi.ll
+++ b/llvm/test/CodeGen/ARM/unaligned_load_store_aeabi.ll
@@ -677,32 +677,34 @@ entry:
define void @store12_align4_trunc_minsize(i96* %a, i96 %b) nounwind minsize {
; CHECK-V6M-MINSIZE-LABEL: store12_align4_trunc_minsize:
; CHECK-V6M-MINSIZE: @ %bb.0: @ %entry
-; CHECK-V6M-MINSIZE-NEXT: .save {r4, lr}
-; CHECK-V6M-MINSIZE-NEXT: push {r4, lr}
-; CHECK-V6M-MINSIZE-NEXT: mov r1, r3
-; CHECK-V6M-MINSIZE-NEXT: mov r4, r0
-; CHECK-V6M-MINSIZE-NEXT: mov r0, r2
-; CHECK-V6M-MINSIZE-NEXT: mov r2, r4
+; CHECK-V6M-MINSIZE-NEXT: .save {r4, r5, r7, lr}
+; CHECK-V6M-MINSIZE-NEXT: push {r4, r5, r7, lr}
+; CHECK-V6M-MINSIZE-NEXT: mov r4, r3
+; CHECK-V6M-MINSIZE-NEXT: mov r5, r0
+; CHECK-V6M-MINSIZE-NEXT: mov r0, r1
+; CHECK-V6M-MINSIZE-NEXT: mov r1, r2
+; CHECK-V6M-MINSIZE-NEXT: mov r2, r5
; CHECK-V6M-MINSIZE-NEXT: bl __aeabi_uwrite8
-; CHECK-V6M-MINSIZE-NEXT: adds r4, #8
-; CHECK-V6M-MINSIZE-NEXT: ldr r0, [sp, #8]
-; CHECK-V6M-MINSIZE-NEXT: mov r1, r4
+; CHECK-V6M-MINSIZE-NEXT: adds r5, #8
+; CHECK-V6M-MINSIZE-NEXT: mov r0, r4
+; CHECK-V6M-MINSIZE-NEXT: mov r1, r5
; CHECK-V6M-MINSIZE-NEXT: bl __aeabi_uwrite4
-; CHECK-V6M-MINSIZE-NEXT: pop {r4, pc}
+; CHECK-V6M-MINSIZE-NEXT: pop {r4, r5, r7, pc}
;
; CHECK-V7M-MINSIZE-LABEL: store12_align4_trunc_minsize:
; CHECK-V7M-MINSIZE: @ %bb.0: @ %entry
-; CHECK-V7M-MINSIZE-NEXT: .save {r4, lr}
-; CHECK-V7M-MINSIZE-NEXT: push {r4, lr}
-; CHECK-V7M-MINSIZE-NEXT: mov r4, r0
-; CHECK-V7M-MINSIZE-NEXT: mov r0, r2
-; CHECK-V7M-MINSIZE-NEXT: mov r1, r3
-; CHECK-V7M-MINSIZE-NEXT: mov r2, r4
+; CHECK-V7M-MINSIZE-NEXT: .save {r4, r5, r7, lr}
+; CHECK-V7M-MINSIZE-NEXT: push {r4, r5, r7, lr}
+; CHECK-V7M-MINSIZE-NEXT: mov r5, r0
+; CHECK-V7M-MINSIZE-NEXT: mov r0, r1
+; CHECK-V7M-MINSIZE-NEXT: mov r1, r2
+; CHECK-V7M-MINSIZE-NEXT: mov r2, r5
+; CHECK-V7M-MINSIZE-NEXT: mov r4, r3
; CHECK-V7M-MINSIZE-NEXT: bl __aeabi_uwrite8
-; CHECK-V7M-MINSIZE-NEXT: ldr r0, [sp, #8]
-; CHECK-V7M-MINSIZE-NEXT: add.w r1, r4, #8
+; CHECK-V7M-MINSIZE-NEXT: add.w r1, r5, #8
+; CHECK-V7M-MINSIZE-NEXT: mov r0, r4
; CHECK-V7M-MINSIZE-NEXT: bl __aeabi_uwrite4
-; CHECK-V7M-MINSIZE-NEXT: pop {r4, pc}
+; CHECK-V7M-MINSIZE-NEXT: pop {r4, r5, r7, pc}
entry:
store i96 %b, i96* %a, align 1
ret void
diff --git a/llvm/test/CodeGen/AVR/llvm.sincos.ll b/llvm/test/CodeGen/AVR/llvm.sincos.ll
index ff01da9fd167b..63dde6cc0d685 100644
--- a/llvm/test/CodeGen/AVR/llvm.sincos.ll
+++ b/llvm/test/CodeGen/AVR/llvm.sincos.ll
@@ -282,14 +282,14 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
; CHECK-NEXT: push r29
; CHECK-NEXT: in r28, 61
; CHECK-NEXT: in r29, 62
-; CHECK-NEXT: sbiw r28, 34
+; CHECK-NEXT: sbiw r28, 54
; CHECK-NEXT: in r0, 63
; CHECK-NEXT: cli
; CHECK-NEXT: out 62, r29
; CHECK-NEXT: out 63, r0
; CHECK-NEXT: out 61, r28
-; CHECK-NEXT: std Y+2, r23 ; 2-byte Folded Spill
-; CHECK-NEXT: std Y+1, r22 ; 2-byte Folded Spill
+; CHECK-NEXT: std Y+16, r23 ; 2-byte Folded Spill
+; CHECK-NEXT: std Y+15, r22 ; 2-byte Folded Spill
; CHECK-NEXT: mov r2, r20
; CHECK-NEXT: mov r3, r21
; CHECK-NEXT: mov r4, r18
@@ -298,85 +298,85 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
; CHECK-NEXT: mov r7, r25
; CHECK-NEXT: mov r24, r28
; CHECK-NEXT: mov r25, r29
-; CHECK-NEXT: adiw r24, 3
+; CHECK-NEXT: adiw r24, 17
; CHECK-NEXT: rcall cosl
; CHECK-NEXT: mov r24, r28
; CHECK-NEXT: mov r25, r29
-; CHECK-NEXT: adiw r24, 19
+; CHECK-NEXT: adiw r24, 33
; CHECK-NEXT: mov r18, r4
; CHECK-NEXT: mov r19, r5
; CHECK-NEXT: mov r20, r2
; CHECK-NEXT: mov r21, r3
-; CHECK-NEXT: ldd r22, Y+1 ; 2-byte Folded Reload
-; CHECK-NEXT: ldd r23, Y+2 ; 2-byte Folded Reload
+; CHECK-NEXT: ldd r22, Y+15 ; 2-byte Folded Reload
+; CHECK-NEXT: ldd r23, Y+16 ; 2-byte Folded Reload
; CHECK-NEXT: rcall sinl
-; CHECK-NEXT: ldd r24, Y+17
-; CHECK-NEXT: ldd r25, Y+18
+; CHECK-NEXT: ldd r24, Y+31
+; CHECK-NEXT: ldd r25, Y+32
; CHECK-NEXT: mov r30, r6
; CHECK-NEXT: mov r31, r7
; CHECK-NEXT: std Z+31, r25
; CHECK-NEXT: std Z+30, r24
-; CHECK-NEXT: ldd r24, Y+15
-; CHECK-NEXT: ldd r25, Y+16
+; CHECK-NEXT: ldd r24, Y+29
+; CHECK-NEXT: ldd r25, Y+30
; CHECK-NEXT: std Z+29, r25
; CHECK-NEXT: std Z+28, r24
-; CHECK-NEXT: ldd r24, Y+13
-; CHECK-NEXT: ldd r25, Y+14
+; CHECK-NEXT: ldd r24, Y+27
+; CHECK-NEXT: ldd r25, Y+28
; CHECK-NEXT: std Z+27, r25
; CHECK-NEXT: std Z+26, r24
-; CHECK-NEXT: ldd r24, Y+11
-; CHECK-NEXT: ldd r25, Y+12
+; CHECK-NEXT: ldd r24, Y+25
+; CHECK-NEXT: ldd r25, Y+26
; CHECK-NEXT: std Z+25, r25
; CHECK-NEXT: std Z+24, r24
-; CHECK-NEXT: ldd r24, Y+9
-; CHECK-NEXT: ldd r25, Y+10
+; CHECK-NEXT: ldd r24, Y+23
+; CHECK-NEXT: ldd r25, Y+24
; CHECK-NEXT: std Z+23, r25
; CHECK-NEXT: std Z+22, r24
-; CHECK-NEXT: ldd r24, Y+7
-; CHECK-NEXT: ldd r25, Y+8
+; CHECK-NEXT: ldd r24, Y+21
+; CHECK-NEXT: ldd r25, Y+22
; CHECK-NEXT: std Z+21, r25
; CHECK-NEXT: std Z+20, r24
-; CHECK-NEXT: ldd r24, Y+5
-; CHECK-NEXT: ldd r25, Y+6
+; CHECK-NEXT: ldd r24, Y+19
+; CHECK-NEXT: ldd r25, Y+20
; CHECK-NEXT: std Z+19, r25
; CHECK-NEXT: std Z+18, r24
-; CHECK-NEXT: ldd r24, Y+3
-; CHECK-NEXT: ldd r25, Y+4
+; CHECK-NEXT: ldd r24, Y+17
+; CHECK-NEXT: ldd r25, Y+18
; CHECK-NEXT: std Z+17, r25
; CHECK-NEXT: std Z+16, r24
-; CHECK-NEXT: ldd r24, Y+33
-; CHECK-NEXT: ldd r25, Y+34
+; CHECK-NEXT: ldd r24, Y+47
+; CHECK-NEXT: ldd r25, Y+48
; CHECK-NEXT: std Z+15, r25
; CHECK-NEXT: std Z+14, r24
-; CHECK-NEXT: ldd r24, Y+31
-; CHECK-NEXT: ldd r25, Y+32
+; CHECK-NEXT: ldd r24, Y+45
+; CHECK-NEXT: ldd r25, Y+46
; CHECK-NEXT: std Z+13, r25
; CHECK-NEXT: std Z+12, r24
-; CHECK-NEXT: ldd r24, Y+29
-; CHECK-NEXT: ldd r25, Y+30
+; CHECK-NEXT: ldd r24, Y+43
+; CHECK-NEXT: ldd r25, Y+44
; CHECK-NEXT: std Z+11, r25
; CHECK-NEXT: std Z+10, r24
-; CHECK-NEXT: ldd r24, Y+27
-; CHECK-NEXT: ldd r25, Y+28
+; CHECK-NEXT: ldd r24, Y+41
+; CHECK-NEXT: ldd r25, Y+42
; CHECK-NEXT: std Z+9, r25
; CHECK-NEXT: std Z+8, r24
-; CHECK-NEXT: ldd r24, Y+25
-; CHECK-NEXT: ldd r25, Y+26
+; CHECK-NEXT: ldd r24, Y+39
+; CHECK-NEXT: ldd r25, Y+40
; CHECK-NEXT: std Z+7, r25
; CHECK-NEXT: std Z+6, r24
-; CHECK-NEXT: ldd r24, Y+23
-; CHECK-NEXT: ldd r25, Y+24
+; CHECK-NEXT: ldd r24, Y+37
+; CHECK-NEXT: ldd r25, Y+38
; CHECK-NEXT: std Z+5, r25
; CHECK-NEXT: std Z+4, r24
-; CHECK-NEXT: ldd r24, Y+21
-; CHECK-NEXT: ldd r25, Y+22
+; CHECK-NEXT: ldd r24, Y+35
+; CHECK-NEXT: ldd r25, Y+36
; CHECK-NEXT: std Z+3, r25
; CHECK-NEXT: std Z+2, r24
-; CHECK-NEXT: ldd r24, Y+19
-; CHECK-NEXT: ldd r25, Y+20
+; CHECK-NEXT: ldd r24, Y+33
+; CHECK-NEXT: ldd r25, Y+34
; CHECK-NEXT: std Z+1, r25
; CHECK-NEXT: st Z, r24
-; CHECK-NEXT: adiw r28, 34
+; CHECK-NEXT: adiw r28, 54
; CHECK-NEXT: in r0, 63
; CHECK-NEXT: cli
; CHECK-NEXT: out 62, r29
diff --git a/llvm/test/CodeGen/Hexagon/llvm.exp10.ll b/llvm/test/CodeGen/Hexagon/llvm.exp10.ll
index cd94d328f1fee..aa6ac4af9d71f 100644
--- a/llvm/test/CodeGen/Hexagon/llvm.exp10.ll
+++ b/llvm/test/CodeGen/Hexagon/llvm.exp10.ll
@@ -145,9 +145,12 @@ define fp128 @exp10_f128(fp128 %x) #0 {
; CHECK-LABEL: exp10_f128:
; CHECK: // %bb.0:
; CHECK-NEXT: {
+; CHECK-NEXT: allocframe(r29,#32):raw
+; CHECK-NEXT: }
+; CHECK-NEXT: {
+; CHECK-NEXT: r29 = and(r29,#-16)
; CHECK-NEXT: r16 = r0
-; CHECK-NEXT: memd(r29+#-16) = r17:16
-; CHECK-NEXT: allocframe(#24)
+; CHECK-NEXT: memd(r30+#-8) = r17:16
; CHECK-NEXT: } // 8-byte Folded Spill
; CHECK-NEXT: {
; CHECK-NEXT: call exp10l
@@ -162,8 +165,8 @@ define fp128 @exp10_f128(fp128 %x) #0 {
; CHECK-NEXT: memd(r16+#0) = r1:0
; CHECK-NEXT: }
; CHECK-NEXT: {
-; CHECK-NEXT: r17:16 = memd(r29+#16)
-; CHECK-NEXT: dealloc_return
+; CHECK-NEXT: r17:16 = memd(r30+#-8)
+; CHECK-NEXT: r31:30 = dealloc_return(r30):raw
; CHECK-NEXT: } // 8-byte Folded Reload
%r = call fp128 @llvm.exp10.f128(fp128 %x)
ret fp128 %r
@@ -173,19 +176,22 @@ define <2 x fp128> @exp10_v2f128(<2 x fp128> %x) #0 {
; CHECK-LABEL: exp10_v2f128:
; CHECK: // %bb.0:
; CHECK-NEXT: {
+; CHECK-NEXT: allocframe(r29,#64):raw
+; CHECK-NEXT: }
+; CHECK-NEXT: {
+; CHECK-NEXT: r29 = and(r29,#-16)
; CHECK-NEXT: r16 = r0
-; CHECK-NEXT: memd(r29+#-16) = r17:16
-; CHECK-NEXT: allocframe(#56)
+; CHECK-NEXT: memd(r30+#-8) = r17:16
+; CHECK-NEXT: memd(r30+#-16) = r19:18
; CHECK-NEXT: } // 8-byte Folded Spill
; CHECK-NEXT: {
; CHECK-NEXT: r0 = add(r29,#16)
-; CHECK-NEXT: memd(r29+#40) = r19:18
-; CHECK-NEXT: memd(r29+#32) = r21:20
-; CHECK-NEXT: } // 8-byte Folded Spill
+; CHECK-NEXT: r19:18 = memd(r30+#8)
+; CHECK-NEXT: memd(r30+#-24) = r21:20
+; CHECK-NEXT: }
; CHECK-NEXT: {
; CHECK-NEXT: call exp10l
-; CHECK-NEXT: r19:18 = memd(r29+#64)
-; CHECK-NEXT: r21:20 = memd(r29+#72)
+; CHECK-NEXT: r21:20 = memd(r30+#16)
; CHECK-NEXT: }
; CHECK-NEXT: {
; CHECK-NEXT: call exp10l
@@ -210,12 +216,12 @@ define <2 x fp128> @exp10_v2f128(<2 x fp128> %x) #0 {
; CHECK-NEXT: memd(r16+#0) = r1:0
; CHECK-NEXT: }
; CHECK-NEXT: {
-; CHECK-NEXT: r17:16 = memd(r29+#48)
-; CHECK-NEXT: r19:18 = memd(r29+#40)
+; CHECK-NEXT: r17:16 = memd(r30+#-8)
+; CHECK-NEXT: r19:18 = memd(r30+#-16)
; CHECK-NEXT: } // 8-byte Folded Reload
; CHECK-NEXT: {
-; CHECK-NEXT: r21:20 = memd(r29+#32)
-; CHECK-NEXT: dealloc_return
+; CHECK-NEXT: r21:20 = memd(r30+#-24)
+; CHECK-NEXT: r31:30 = dealloc_return(r30):raw
; CHECK-NEXT: } // 8-byte Folded Reload
%r = call <2 x fp128> @llvm.exp10.v2f128(<2 x fp128> %x)
ret <2 x fp128> %r
diff --git a/llvm/test/CodeGen/Hexagon/llvm.sincos.ll b/llvm/test/CodeGen/Hexagon/llvm.sincos.ll
index f02ac2ca8480f..f1ecae178abb5 100644
--- a/llvm/test/CodeGen/Hexagon/llvm.sincos.ll
+++ b/llvm/test/CodeGen/Hexagon/llvm.sincos.ll
@@ -1097,19 +1097,22 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
; BASE-LABEL: test_sincos_f128:
; BASE: // %bb.0:
; BASE-NEXT: {
-; BASE-NEXT: r17:16 = combine(r5,r4)
-; BASE-NEXT: memd(r29+#-16) = r17:16
-; BASE-NEXT: allocframe(#56)
-; BASE-NEXT: } // 8-byte Folded Spill
+; BASE-NEXT: allocframe(r29,#64):raw
+; BASE-NEXT: }
; BASE-NEXT: {
+; BASE-NEXT: r29 = and(r29,#-16)
; BASE-NEXT: r20 = r0
+; BASE-NEXT: memd(r30+#-24) = r21:20
+; BASE-NEXT: memd(r30+#-8) = r17:16
+; BASE-NEXT: } // 8-byte Folded Spill
+; BASE-NEXT: {
; BASE-NEXT: r0 = add(r29,#0)
-; BASE-NEXT: memd(r29+#32) = r21:20
-; BASE-NEXT: memd(r29+#40) = r19:18
+; BASE-NEXT: r17:16 = combine(r5,r4)
+; BASE-NEXT: r19:18 = combine(r3,r2)
+; BASE-NEXT: memd(r30+#-16) = r19:18
; BASE-NEXT: } // 8-byte Folded Spill
; BASE-NEXT: {
; BASE-NEXT: call sinl
-; BASE-NEXT: r19:18 = combine(r3,r2)
; BASE-NEXT: }
; BASE-NEXT: {
; BASE-NEXT: call cosl
@@ -1134,25 +1137,28 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
; BASE-NEXT: memd(r20+#0) = r1:0
; BASE-NEXT: }
; BASE-NEXT: {
-; BASE-NEXT: r17:16 = memd(r29+#48)
-; BASE-NEXT: r19:18 = memd(r29+#40)
+; BASE-NEXT: r17:16 = memd(r30+#-8)
+; BASE-NEXT: r19:18 = memd(r30+#-16)
; BASE-NEXT: } // 8-byte Folded Reload
; BASE-NEXT: {
-; BASE-NEXT: r21:20 = memd(r29+#32)
-; BASE-NEXT: dealloc_return
+; BASE-NEXT: r21:20 = memd(r30+#-24)
+; BASE-NEXT: r31:30 = dealloc_return(r30):raw
; BASE-NEXT: } // 8-byte Folded Reload
;
; GNU-LABEL: test_sincos_f128:
; GNU: // %bb.0:
; GNU-NEXT: {
+; GNU-NEXT: allocframe(r29,#80):raw
+; GNU-NEXT: }
+; GNU-NEXT: {
+; GNU-NEXT: r29 = and(r29,#-16)
; GNU-NEXT: r16 = r0
-; GNU-NEXT: memd(r29+#-16) = r17:16
-; GNU-NEXT: allocframe(#64)
+; GNU-NEXT: memd(r30+#-8) = r17:16
; GNU-NEXT: } // 8-byte Folded Spill
; GNU-NEXT: {
-; GNU-NEXT: r0 = add(r29,#40)
-; GNU-NEXT: r7 = add(r29,#24)
-; GNU-NEXT: r6 = add(r29,#8)
+; GNU-NEXT: r0 = add(r29,#48)
+; GNU-NEXT: r7 = add(r29,#32)
+; GNU-NEXT: r6 = add(r29,#16)
; GNU-NEXT: memw(r29+#4) = r7.new
; GNU-NEXT: }
; GNU-NEXT: {
@@ -1160,12 +1166,12 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
; GNU-NEXT: memw(r29+#0) = r6
; GNU-NEXT: }
; GNU-NEXT: {
-; GNU-NEXT: r1:0 = memd(r29+#8)
-; GNU-NEXT: r3:2 = memd(r29+#16)
+; GNU-NEXT: r1:0 = memd(r29+#16)
+; GNU-NEXT: r3:2 = memd(r29+#24)
; GNU-NEXT: }
; GNU-NEXT: {
-; GNU-NEXT: r5:4 = memd(r29+#24)
-; GNU-NEXT: r7:6 = memd(r29+#32)
+; GNU-NEXT: r5:4 = memd(r29+#32)
+; GNU-NEXT: r7:6 = memd(r29+#40)
; GNU-NEXT: }
; GNU-NEXT: {
; GNU-NEXT: memd(r16+#24) = r7:6
@@ -1176,26 +1182,29 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
; GNU-NEXT: memd(r16+#0) = r1:0
; GNU-NEXT: }
; GNU-NEXT: {
-; GNU-NEXT: r17:16 = memd(r29+#56)
-; GNU-NEXT: dealloc_return
+; GNU-NEXT: r17:16 = memd(r30+#-8)
+; GNU-NEXT: r31:30 = dealloc_return(r30):raw
; GNU-NEXT: } // 8-byte Folded Reload
;
; MUSL-LABEL: test_sincos_f128:
; MUSL: // %bb.0:
; MUSL-NEXT: {
-; MUSL-NEXT: r17:16 = combine(r5,r4)
-; MUSL-NEXT: memd(r29+#-16) = r17:16
-; MUSL-NEXT: allocframe(#56)
-; MUSL-NEXT: } // 8-byte Folded Spill
+; MUSL-NEXT: allocframe(r29,#64):raw
+; MUSL-NEXT: }
; MUSL-NEXT: {
+; MUSL-NEXT: r29 = and(r29,#-16)
; MUSL-NEXT: r20 = r0
+; MUSL-NEXT: memd(r30+#-24) = r21:20
+; MUSL-NEXT: memd(r30+#-8) = r17:16
+; MUSL-NEXT: } // 8-byte Folded Spill
+; MUSL-NEXT: {
; MUSL-NEXT: r0 = add(r29,#0)
-; MUSL-NEXT: memd(r29+#32) = r21:20
-; MUSL-NEXT: memd(r29+#40) = r19:18
+; MUSL-NEXT: r17:16 = combine(r5,r4)
+; MUSL-NEXT: r19:18 = combine(r3,r2)
+; MUSL-NEXT: memd(r30+#-16) = r19:18
; MUSL-NEXT: } // 8-byte Folded Spill
; MUSL-NEXT: {
; MUSL-NEXT: call sinl
-; MUSL-NEXT: r19:18 = combine(r3,r2)
; MUSL-NEXT: }
; MUSL-NEXT: {
; MUSL-NEXT: call cosl
@@ -1220,12 +1229,12 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
; MUSL-NEXT: memd(r20+#0) = r1:0
; MUSL-NEXT: }
; MUSL-NEXT: {
-; MUSL-NEXT: r17:16 = memd(r29+#48)
-; MUSL-NEXT: r19:18 = memd(r29+#40)
+; MUSL-NEXT: r17:16 = memd(r30+#-8)
+; MUSL-NEXT: r19:18 = memd(r30+#-16)
; MUSL-NEXT: } // 8-byte Folded Reload
; MUSL-NEXT: {
-; MUSL-NEXT: r21:20 = memd(r29+#32)
-; MUSL-NEXT: dealloc_return
+; MUSL-NEXT: r21:20 = memd(r30+#-24)
+; MUSL-NEXT: r31:30 = dealloc_return(r30):raw
; MUSL-NEXT: } // 8-byte Folded Reload
%result = call { fp128, fp128 } @llvm.sincos.f128(fp128 %a)
ret { fp128, fp128 } %result
@@ -1235,25 +1244,28 @@ define { <2 x fp128>, <2 x fp128> } @test_sincos_v2f128(<2 x fp128> %a) #0 {
; BASE-LABEL: test_sincos_v2f128:
; BASE: // %bb.0:
; BASE-NEXT: {
-; BASE-NEXT: r17:16 = combine(r5,r4)
-; BASE-NEXT: memd(r29+#-16) = r17:16
-; BASE-NEXT: allocframe(#104)
-; BASE-NEXT: } // 8-byte Folded Spill
+; BASE-NEXT: allocframe(r29,#112):raw
+; BASE-NEXT: }
; BASE-NEXT: {
+; BASE-NEXT: r29 = and(r29,#-16)
; BASE-NEXT: r20 = r0
+; BASE-NEXT: memd(r30+#-24) = r21:20
+; BASE-NEXT: memd(r30+#-8) = r17:16
+; BASE-NEXT: } // 8-byte Folded Spill
+; BASE-NEXT: {
; BASE-NEXT: r0 = add(r29,#32)
-; BASE-NEXT: memd(r29+#80) = r21:20
-; BASE-NEXT: memd(r29+#88) = r19:18
+; BASE-NEXT: r17:16 = combine(r5,r4)
+; BASE-NEXT: memd(r30+#-16) = r19:18
+; BASE-NEXT: memd(r30+#-32) = r23:22
; BASE-NEXT: } // 8-byte Folded Spill
; BASE-NEXT: {
; BASE-NEXT: r19:18 = combine(r3,r2)
-; BASE-NEXT: memd(r29+#72) = r23:22
-; BASE-NEXT: memd(r29+#64) = r25:24
-; BASE-NEXT: } // 8-byte Folded Spill
+; BASE-NEXT: r23:22 = memd(r30+#8)
+; BASE-NEXT: memd(r30+#-40) = r25:24
+; BASE-NEXT: }
; BASE-NEXT: {
; BASE-NEXT: call sinl
-; BASE-NEXT: r23:22 = memd(r29+#112)
-; BASE-NEXT: r25:24 = memd(r29+#120)
+; BASE-NEXT: r25:24 = memd(r30+#16)
; BASE-NEXT: }
; BASE-NEXT: {
; BASE-NEXT: call sinl
@@ -1306,69 +1318,72 @@ define { <2 x fp128>, <2 x fp128> } @test_sincos_v2f128(<2 x fp128> %a) #0 {
; BASE-NEXT: memd(r20+#0) = r1:0
; BASE-NEXT: }
; BASE-NEXT: {
-; BASE-NEXT: r17:16 = memd(r29+#96)
-; BASE-NEXT: r19:18 = memd(r29+#88)
+; BASE-NEXT: r17:16 = memd(r30+#-8)
+; BASE-NEXT: r19:18 = memd(r30+#-16)
; BASE-NEXT: } // 8-byte Folded Reload
; BASE-NEXT: {
-; BASE-NEXT: r21:20 = memd(r29+#80)
-; BASE-NEXT: r23:22 = memd(r29+#72)
+; BASE-NEXT: r21:20 = memd(r30+#-24)
+; BASE-NEXT: r23:22 = memd(r30+#-32)
; BASE-NEXT: } // 8-byte Folded Reload
; BASE-NEXT: {
-; BASE-NEXT: r25:24 = memd(r29+#64)
+; BASE-NEXT: r25:24 = memd(r30+#-40)
; BASE-NEXT: r31:30 = dealloc_return(r30):raw
; BASE-NEXT: } // 8-byte Folded Reload
;
; GNU-LABEL: test_sincos_v2f128:
; GNU: // %bb.0:
; GNU-NEXT: {
+; GNU-NEXT: allocframe(r29,#144):raw
+; GNU-NEXT: }
+; GNU-NEXT: {
+; GNU-NEXT: r29 = and(r29,#-16)
; GNU-NEXT: r16 = r0
-; GNU-NEXT: memd(r29+#-16) = r17:16
-; GNU-NEXT: allocframe(#128)
+; GNU-NEXT: memd(r30+#-8) = r17:16
+; GNU-NEXT: memd(r30+#-16) = r19:18
; GNU-NEXT: } // 8-byte Folded Spill
; GNU-NEXT: {
-; GNU-NEXT: r0 = add(r29,#88)
+; GNU-NEXT: r0 = add(r29,#96)
; GNU-NEXT: r17 = r29
-; GNU-NEXT: memd(r29+#112) = r19:18
-; GNU-NEXT: memd(r29+#104) = r21:20
+; GNU-NEXT: r6 = add(r29,#64)
+; GNU-NEXT: memd(r30+#-24) = r21:20
; GNU-NEXT: } // 8-byte Folded Spill
; GNU-NEXT: {
-; GNU-NEXT: r6 = add(r29,#56)
-; GNU-NEXT: r1 = add(r29,#72)
-; GNU-NEXT: r19:18 = memd(r29+#136)
-; GNU-NEXT: memw(r17+#0) = r6.new
+; GNU-NEXT: r1 = add(r29,#80)
+; GNU-NEXT: r19:18 = memd(r30+#8)
+; GNU-NEXT: memw(r17+#0) = r6
; GNU-NEXT: }
; GNU-NEXT: {
; GNU-NEXT: call sincosl
-; GNU-NEXT: r21:20 = memd(r29+#144)
+; GNU-NEXT: r21:20 = memd(r30+#16)
; GNU-NEXT: memw(r17+#4) = r1
; GNU-NEXT: }
; GNU-NEXT: {
-; GNU-NEXT: r0 = add(r29,#24)
+; GNU-NEXT: r0 = add(r29,#32)
; GNU-NEXT: r3:2 = combine(r19,r18)
; GNU-NEXT: r5:4 = combine(r21,r20)
; GNU-NEXT: memw(r17+#4) = r0.new
; GNU-NEXT: }
; GNU-NEXT: {
; GNU-NEXT: call sincosl
-; GNU-NEXT: r0 = add(r29,#40)
-; GNU-NEXT: r1 = add(r29,#8)
+; GNU-NEXT: r0 = add(r29,#48)
+; GNU-NEXT: r1 = add(r29,#16)
; GNU-NEXT: memw(r17+#0) = r1.new
; GNU-NEXT: }
; GNU-NEXT: {
-; GNU-NEXT: r1:0 = memd(r29+#56)
-; GNU-NEXT: r3:2 = memd(r29+#64)
+; GNU-NEXT: r1:0 = memd(r29+#64)
+; GNU-NEXT: r3:2 = memd(r29+#72)
; GNU-NEXT: }
; GNU-NEXT: {
-; GNU-NEXT: r5:4 = memd(r29+#8)
-; GNU-NEXT: r7:6 = memd(r29+#16)
+; GNU-NEXT: r5:4 = memd(r29+#16)
+; GNU-NEXT: r7:6 = memd(r29+#24)
; GNU-NEXT: }
; GNU-NEXT: {
-; GNU-NEXT: r9:8 = memd(r29+#72)
-; GNU-NEXT: r13:12 = memd(r29+#24)
+; GNU-NEXT: r9:8 = memd(r29+#80)
+; GNU-NEXT: r13:12 = memd(r29+#32)
; GNU-NEXT: }
; GNU-NEXT: {
-; GNU-NEXT: r15:14 = memd(r29+#32)
-; GNU-NEXT: r11:10 = memd(r29+#80)
+; GNU-NEXT: r15:14 = memd(r29+#40)
+; GNU-NEXT: r11:10 = memd(r29+#88)
; GNU-NEXT: }
; GNU-NEXT: {
; GNU-NEXT: memd(r16+#56) = r15:14
@@ -1387,36 +1402,39 @@ define { <2 x fp128>, <2 x fp128> } @test_sincos_v2f128(<2 x fp128> %a) #0 {
; GNU-NEXT: memd(r16+#0) = r1:0
; GNU-NEXT: }
; GNU-NEXT: {
-; GNU-NEXT: r17:16 = memd(r29+#120)
-; GNU-NEXT: r19:18 = memd(r29+#112)
+; GNU-NEXT: r17:16 = memd(r30+#-8)
+; GNU-NEXT: r19:18 = memd(r30+#-16)
; GNU-NEXT: } // 8-byte Folded Reload
; GNU-NEXT: {
-; GNU-NEXT: r21:20 = memd(r29+#104)
-; GNU-NEXT: dealloc_return
+; GNU-NEXT: r21:20 = memd(r30+#-24)
+; GNU-NEXT: r31:30 = dealloc_return(r30):raw
; GNU-NEXT: } // 8-byte Folded Reload
;
; MUSL-LABEL: test_sincos_v2f128:
; MUSL: // %bb.0:
; MUSL-NEXT: {
-; MUSL-NEXT: r17:16 = combine(r5,r4)
-; MUSL-NEXT: memd(r29+#-16) = r17:16
-; MUSL-NEXT: allocframe(#104)
-; MUSL-NEXT: } // 8-byte Folded Spill
+; MUSL-NEXT: allocframe(r29,#112):raw
+; MUSL-NEXT: }
; MUSL-NEXT: {
+; MUSL-NEXT: r29 = and(r29,#-16)
; MUSL-NEXT: r20 = r0
+; MUSL-NEXT: memd(r30+#-24) = r21:20
+; MUSL-NEXT: memd(r30+#-8) = r17:16
+; MUSL-NEXT: } // 8-byte Folded Spill
+; MUSL-NEXT: {
; MUSL-NEXT: r0 = add(r29,#32)
-; MUSL-NEXT: memd(r29+#80) = r21:20
-; MUSL-NEXT: memd(r29+#88) = r19:18
+; MUSL-NEXT: r17:16 = combine(r5,r4)
+; MUSL-NEXT: memd(r30+#-16) = r19:18
+; MUSL-NEXT: memd(r30+#-32) = r23:22
; MUSL-NEXT: } // 8-byte Folded Spill
; MUSL-NEXT: {
; MUSL-NEXT: r19:18 = combine(r3,r2)
-; MUSL-NEXT: memd(r29+#72) = r23:22
-; MUSL-NEXT: memd(r29+#64) = r25:24
-; MUSL-NEXT: } // 8-byte Folded Spill
+; MUSL-NEXT: r23:22 = memd(r30+#8)
+; MUSL-NEXT: memd(r30+#-40) = r25:24
+; MUSL-NEXT: }
; MUSL-NEXT: {
; MUSL-NEXT: call sinl
-; MUSL-NEXT: r23:22 = memd(r29+#112)
-; MUSL-NEXT: r25:24 = memd(r29+#120)
+; MUSL-NEXT: r25:24 = memd(r30+#16)
; MUSL-NEXT: }
; MUSL-NEXT: {
; MUSL-NEXT: call sinl
@@ -1469,15 +1487,15 @@ define { <2 x fp128>, <2 x fp128> } @test_sincos_v2f128(<2 x fp128> %a) #0 {
; MUSL-NEXT: memd(r20+#0) = r1:0
; MUSL-NEXT: }
; MUSL-NEXT: {
-; MUSL-NEXT: r17:16 = memd(r29+#96)
-; MUSL-NEXT: r19:18 = memd(r29+#88)
+; MUSL-NEXT: r17:16 = memd(r30+#-8)
+; MUSL-NEXT: r19:18 = memd(r30+#-16)
; MUSL-NEXT: } // 8-byte Folded Reload
; MUSL-NEXT: {
-; MUSL-NEXT: r21:20 = memd(r29+#80)
-; MUSL-NEXT: r23:22 = memd(r29+#72)
+; MUSL-NEXT: r21:20 = memd(r30+#-24)
+; MUSL-NEXT: r23:22 = memd(r30+#-32)
; MUSL-NEXT: } // 8-byte Folded Reload
; MUSL-NEXT: {
-; MUSL-NEXT: r25:24 = memd(r29+#64)
+; MUSL-NEXT: r25:24 = memd(r30+#-40)
; MUSL-NEXT: r31:30 = dealloc_return(r30):raw
; MUSL-NEXT: } // 8-byte Folded Reload
%result = call { <2 x fp128>, <2 x fp128> } @llvm.sincos.v2f128(<2 x fp128> %a)
diff --git a/llvm/test/CodeGen/LoongArch/llvm.exp10.ll b/llvm/test/CodeGen/LoongArch/llvm.exp10.ll
index 030b822c4c45d..21cf4e936e29b 100644
--- a/llvm/test/CodeGen/LoongArch/llvm.exp10.ll
+++ b/llvm/test/CodeGen/LoongArch/llvm.exp10.ll
@@ -228,17 +228,17 @@ define fp128 @exp10_f128(fp128 %x) #0 {
; LA32-NEXT: ld.w $a4, $a1, 8
; LA32-NEXT: ld.w $a1, $a1, 12
; LA32-NEXT: move $fp, $a0
-; LA32-NEXT: st.w $a1, $sp, 20
-; LA32-NEXT: st.w $a4, $sp, 16
-; LA32-NEXT: st.w $a3, $sp, 12
-; LA32-NEXT: addi.w $a0, $sp, 24
-; LA32-NEXT: addi.w $a1, $sp, 8
-; LA32-NEXT: st.w $a2, $sp, 8
+; LA32-NEXT: st.w $a1, $sp, 12
+; LA32-NEXT: st.w $a4, $sp, 8
+; LA32-NEXT: st.w $a3, $sp, 4
+; LA32-NEXT: addi.w $a0, $sp, 16
+; LA32-NEXT: addi.w $a1, $sp, 0
+; LA32-NEXT: st.w $a2, $sp, 0
; LA32-NEXT: bl exp10l
-; LA32-NEXT: ld.w $a0, $sp, 36
-; LA32-NEXT: ld.w $a1, $sp, 32
-; LA32-NEXT: ld.w $a2, $sp, 28
-; LA32-NEXT: ld.w $a3, $sp, 24
+; LA32-NEXT: ld.w $a0, $sp, 28
+; LA32-NEXT: ld.w $a1, $sp, 24
+; LA32-NEXT: ld.w $a2, $sp, 20
+; LA32-NEXT: ld.w $a3, $sp, 16
; LA32-NEXT: st.w $a0, $fp, 12
; LA32-NEXT: st.w $a1, $fp, 8
; LA32-NEXT: st.w $a2, $fp, 4
@@ -280,28 +280,28 @@ define <2 x fp128> @exp10_v2f128(<2 x fp128> %x) #0 {
; LA32-NEXT: ld.w $a4, $a1, 8
; LA32-NEXT: ld.w $a1, $a1, 12
; LA32-NEXT: move $fp, $a0
-; LA32-NEXT: st.w $a1, $sp, 20
-; LA32-NEXT: st.w $a4, $sp, 16
-; LA32-NEXT: st.w $a3, $sp, 12
-; LA32-NEXT: addi.w $a0, $sp, 24
-; LA32-NEXT: addi.w $a1, $sp, 8
-; LA32-NEXT: st.w $a2, $sp, 8
+; LA32-NEXT: st.w $a1, $sp, 12
+; LA32-NEXT: st.w $a4, $sp, 8
+; LA32-NEXT: st.w $a3, $sp, 4
+; LA32-NEXT: addi.w $a0, $sp, 16
+; LA32-NEXT: addi.w $a1, $sp, 0
+; LA32-NEXT: st.w $a2, $sp, 0
; LA32-NEXT: bl exp10l
-; LA32-NEXT: st.w $s3, $sp, 52
-; LA32-NEXT: st.w $s2, $sp, 48
-; LA32-NEXT: st.w $s1, $sp, 44
-; LA32-NEXT: addi.w $a0, $sp, 56
-; LA32-NEXT: addi.w $a1, $sp, 40
-; LA32-NEXT: st.w $s0, $sp, 40
+; LA32-NEXT: st.w $s3, $sp, 44
+; LA32-NEXT: st.w $s2, $sp, 40
+; LA32-NEXT: st.w $s1, $sp, 36
+; LA32-NEXT: addi.w $a0, $sp, 48
+; LA32-NEXT: addi.w $a1, $sp, 32
+; LA32-NEXT: st.w $s0, $sp, 32
; LA32-NEXT: bl exp10l
-; LA32-NEXT: ld.w $a0, $sp, 24
-; LA32-NEXT: ld.w $a1, $sp, 28
-; LA32-NEXT: ld.w $a2, $sp, 32
-; LA32-NEXT: ld.w $a3, $sp, 36
-; LA32-NEXT: ld.w $a4, $sp, 68
-; LA32-NEXT: ld.w $a5, $sp, 64
-; LA32-NEXT: ld.w $a6, $sp, 60
-; LA32-NEXT: ld.w $a7, $sp, 56
+; LA32-NEXT: ld.w $a0, $sp, 16
+; LA32-NEXT: ld.w $a1, $sp, 20
+; LA32-NEXT: ld.w $a2, $sp, 24
+; LA32-NEXT: ld.w $a3, $sp, 28
+; LA32-NEXT: ld.w $a4, $sp, 60
+; LA32-NEXT: ld.w $a5, $sp, 56
+; LA32-NEXT: ld.w $a6, $sp, 52
+; LA32-NEXT: ld.w $a7, $sp, 48
; LA32-NEXT: st.w $a4, $fp, 28
; LA32-NEXT: st.w $a5, $fp, 24
; LA32-NEXT: st.w $a6, $fp, 20
diff --git a/llvm/test/CodeGen/LoongArch/llvm.sincos.ll b/llvm/test/CodeGen/LoongArch/llvm.sincos.ll
index 4ac38a990b664..4fc64bf4a60e7 100644
--- a/llvm/test/CodeGen/LoongArch/llvm.sincos.ll
+++ b/llvm/test/CodeGen/LoongArch/llvm.sincos.ll
@@ -636,28 +636,28 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
; LA32-NEXT: ld.w $s2, $a1, 8
; LA32-NEXT: ld.w $s3, $a1, 12
; LA32-NEXT: move $fp, $a0
-; LA32-NEXT: st.w $s3, $sp, 52
-; LA32-NEXT: st.w $s2, $sp, 48
-; LA32-NEXT: st.w $s1, $sp, 44
-; LA32-NEXT: addi.w $a0, $sp, 56
-; LA32-NEXT: addi.w $a1, $sp, 40
-; LA32-NEXT: st.w $s0, $sp, 40
+; LA32-NEXT: st.w $s3, $sp, 44
+; LA32-NEXT: st.w $s2, $sp, 40
+; LA32-NEXT: st.w $s1, $sp, 36
+; LA32-NEXT: addi.w $a0, $sp, 48
+; LA32-NEXT: addi.w $a1, $sp, 32
+; LA32-NEXT: st.w $s0, $sp, 32
; LA32-NEXT: bl sinl
-; LA32-NEXT: st.w $s3, $sp, 20
-; LA32-NEXT: st.w $s2, $sp, 16
-; LA32-NEXT: st.w $s1, $sp, 12
-; LA32-NEXT: addi.w $a0, $sp, 24
-; LA32-NEXT: addi.w $a1, $sp, 8
-; LA32-NEXT: st.w $s0, $sp, 8
+; LA32-NEXT: st.w $s3, $sp, 12
+; LA32-NEXT: st.w $s2, $sp, 8
+; LA32-NEXT: st.w $s1, $sp, 4
+; LA32-NEXT: addi.w $a0, $sp, 16
+; LA32-NEXT: addi.w $a1, $sp, 0
+; LA32-NEXT: st.w $s0, $sp, 0
; LA32-NEXT: bl cosl
-; LA32-NEXT: ld.w $a0, $sp, 56
-; LA32-NEXT: ld.w $a1, $sp, 60
-; LA32-NEXT: ld.w $a2, $sp, 64
-; LA32-NEXT: ld.w $a3, $sp, 68
-; LA32-NEXT: ld.w $a4, $sp, 36
-; LA32-NEXT: ld.w $a5, $sp, 32
-; LA32-NEXT: ld.w $a6, $sp, 28
-; LA32-NEXT: ld.w $a7, $sp, 24
+; LA32-NEXT: ld.w $a0, $sp, 48
+; LA32-NEXT: ld.w $a1, $sp, 52
+; LA32-NEXT: ld.w $a2, $sp, 56
+; LA32-NEXT: ld.w $a3, $sp, 60
+; LA32-NEXT: ld.w $a4, $sp, 28
+; LA32-NEXT: ld.w $a5, $sp, 24
+; LA32-NEXT: ld.w $a6, $sp, 20
+; LA32-NEXT: ld.w $a7, $sp, 16
; LA32-NEXT: st.w $a4, $fp, 28
; LA32-NEXT: st.w $a5, $fp, 24
; LA32-NEXT: st.w $a6, $fp, 20
@@ -736,50 +736,50 @@ define { <2 x fp128>, <2 x fp128> } @test_sincos_v2f128(<2 x fp128> %a) #0 {
; LA32-NEXT: ld.w $s6, $a1, 8
; LA32-NEXT: ld.w $s7, $a1, 12
; LA32-NEXT: move $fp, $a0
-; LA32-NEXT: st.w $s7, $sp, 52
-; LA32-NEXT: st.w $s6, $sp, 48
-; LA32-NEXT: st.w $s5, $sp, 44
-; LA32-NEXT: addi.w $a0, $sp, 56
-; LA32-NEXT: addi.w $a1, $sp, 40
-; LA32-NEXT: st.w $s4, $sp, 40
+; LA32-NEXT: st.w $s7, $sp, 44
+; LA32-NEXT: st.w $s6, $sp, 40
+; LA32-NEXT: st.w $s5, $sp, 36
+; LA32-NEXT: addi.w $a0, $sp, 48
+; LA32-NEXT: addi.w $a1, $sp, 32
+; LA32-NEXT: st.w $s4, $sp, 32
; LA32-NEXT: bl sinl
-; LA32-NEXT: st.w $s3, $sp, 116
-; LA32-NEXT: st.w $s2, $sp, 112
-; LA32-NEXT: st.w $s1, $sp, 108
-; LA32-NEXT: addi.w $a0, $sp, 120
-; LA32-NEXT: addi.w $a1, $sp, 104
-; LA32-NEXT: st.w $s0, $sp, 104
+; LA32-NEXT: st.w $s3, $sp, 108
+; LA32-NEXT: st.w $s2, $sp, 104
+; LA32-NEXT: st.w $s1, $sp, 100
+; LA32-NEXT: addi.w $a0, $sp, 112
+; LA32-NEXT: addi.w $a1, $sp, 96
+; LA32-NEXT: st.w $s0, $sp, 96
; LA32-NEXT: bl sinl
-; LA32-NEXT: st.w $s7, $sp, 20
-; LA32-NEXT: st.w $s6, $sp, 16
-; LA32-NEXT: st.w $s5, $sp, 12
-; LA32-NEXT: addi.w $a0, $sp, 24
-; LA32-NEXT: addi.w $a1, $sp, 8
-; LA32-NEXT: st.w $s4, $sp, 8
+; LA32-NEXT: st.w $s7, $sp, 12
+; LA32-NEXT: st.w $s6, $sp, 8
+; LA32-NEXT: st.w $s5, $sp, 4
+; LA32-NEXT: addi.w $a0, $sp, 16
+; LA32-NEXT: addi.w $a1, $sp, 0
+; LA32-NEXT: st.w $s4, $sp, 0
; LA32-NEXT: bl cosl
-; LA32-NEXT: st.w $s3, $sp, 84
-; LA32-NEXT: st.w $s2, $sp, 80
-; LA32-NEXT: st.w $s1, $sp, 76
-; LA32-NEXT: addi.w $a0, $sp, 88
-; LA32-NEXT: addi.w $a1, $sp, 72
-; LA32-NEXT: st.w $s0, $sp, 72
+; LA32-NEXT: st.w $s3, $sp, 76
+; LA32-NEXT: st.w $s2, $sp, 72
+; LA32-NEXT: st.w $s1, $sp, 68
+; LA32-NEXT: addi.w $a0, $sp, 80
+; LA32-NEXT: addi.w $a1, $sp, 64
+; LA32-NEXT: st.w $s0, $sp, 64
; LA32-NEXT: bl cosl
-; LA32-NEXT: ld.w $a0, $sp, 56
-; LA32-NEXT: ld.w $a1, $sp, 60
-; LA32-NEXT: ld.w $a2, $sp, 64
-; LA32-NEXT: ld.w $a3, $sp, 68
-; LA32-NEXT: ld.w $a4, $sp, 120
-; LA32-NEXT: ld.w $a5, $sp, 124
-; LA32-NEXT: ld.w $a6, $sp, 128
-; LA32-NEXT: ld.w $a7, $sp, 132
-; LA32-NEXT: ld.w $t0, $sp, 24
-; LA32-NEXT: ld.w $t1, $sp, 28
-; LA32-NEXT: ld.w $t2, $sp, 32
-; LA32-NEXT: ld.w $t3, $sp, 36
-; LA32-NEXT: ld.w $t4, $sp, 100
-; LA32-NEXT: ld.w $t5, $sp, 96
-; LA32-NEXT: ld.w $t6, $sp, 92
-; LA32-NEXT: ld.w $t7, $sp, 88
+; LA32-NEXT: ld.w $a0, $sp, 48
+; LA32-NEXT: ld.w $a1, $sp, 52
+; LA32-NEXT: ld.w $a2, $sp, 56
+; LA32-NEXT: ld.w $a3, $sp, 60
+; LA32-NEXT: ld.w $a4, $sp, 112
+; LA32-NEXT: ld.w $a5, $sp, 116
+; LA32-NEXT: ld.w $a6, $sp, 120
+; LA32-NEXT: ld.w $a7, $sp, 124
+; LA32-NEXT: ld.w $t0, $sp, 16
+; LA32-NEXT: ld.w $t1, $sp, 20
+; LA32-NEXT: ld.w $t2, $sp, 24
+; LA32-NEXT: ld.w $t3, $sp, 28
+; LA32-NEXT: ld.w $t4, $sp, 92
+; LA32-NEXT: ld.w $t5, $sp, 88
+; LA32-NEXT: ld.w $t6, $sp, 84
+; LA32-NEXT: ld.w $t7, $sp, 80
; LA32-NEXT: st.w $t4, $fp, 60
; LA32-NEXT: st.w $t5, $fp, 56
; LA32-NEXT: st.w $t6, $fp, 52
diff --git a/llvm/test/CodeGen/LoongArch/musttail-indirect-args.ll b/llvm/test/CodeGen/LoongArch/musttail-indirect-args.ll
index d088d6065aa07..8e22170d21a93 100644
--- a/llvm/test/CodeGen/LoongArch/musttail-indirect-args.ll
+++ b/llvm/test/CodeGen/LoongArch/musttail-indirect-args.ll
@@ -346,22 +346,22 @@ define i32 @caller_musttail_computed(fp128 %a) nounwind {
; LA32-NEXT: ld.w $a0, $a0, 8
; LA32-NEXT: ld.w $a1, $fp, 12
; LA32-NEXT: ld.w $a2, $fp, 0
-; LA32-NEXT: st.w $a2, $sp, 8
-; LA32-NEXT: st.w $a2, $sp, 24
-; LA32-NEXT: st.w $a1, $sp, 20
-; LA32-NEXT: st.w $a0, $sp, 16
-; LA32-NEXT: st.w $a3, $sp, 12
-; LA32-NEXT: st.w $a1, $sp, 36
-; LA32-NEXT: st.w $a0, $sp, 32
-; LA32-NEXT: addi.w $a0, $sp, 40
-; LA32-NEXT: addi.w $a1, $sp, 24
-; LA32-NEXT: addi.w $a2, $sp, 8
-; LA32-NEXT: st.w $a3, $sp, 28
+; LA32-NEXT: st.w $a2, $sp, 0
+; LA32-NEXT: st.w $a2, $sp, 16
+; LA32-NEXT: st.w $a1, $sp, 12
+; LA32-NEXT: st.w $a0, $sp, 8
+; LA32-NEXT: st.w $a3, $sp, 4
+; LA32-NEXT: st.w $a1, $sp, 28
+; LA32-NEXT: st.w $a0, $sp, 24
+; LA32-NEXT: addi.w $a0, $sp, 32
+; LA32-NEXT: addi.w $a1, $sp, 16
+; LA32-NEXT: addi.w $a2, $sp, 0
+; LA32-NEXT: st.w $a3, $sp, 20
; LA32-NEXT: bl __addtf3
-; LA32-NEXT: ld.w $a0, $sp, 40
-; LA32-NEXT: ld.w $a1, $sp, 44
-; LA32-NEXT: ld.w $a2, $sp, 48
-; LA32-NEXT: ld.w $a3, $sp, 52
+; LA32-NEXT: ld.w $a0, $sp, 32
+; LA32-NEXT: ld.w $a1, $sp, 36
+; LA32-NEXT: ld.w $a2, $sp, 40
+; LA32-NEXT: ld.w $a3, $sp, 44
; LA32-NEXT: st.w $a0, $fp, 0
; LA32-NEXT: st.w $a1, $fp, 4
; LA32-NEXT: st.w $a2, $fp, 8
diff --git a/llvm/test/CodeGen/LoongArch/soft-fp-to-int.ll b/llvm/test/CodeGen/LoongArch/soft-fp-to-int.ll
index e7f75cddc3ec9..bf207f0129088 100644
--- a/llvm/test/CodeGen/LoongArch/soft-fp-to-int.ll
+++ b/llvm/test/CodeGen/LoongArch/soft-fp-to-int.ll
@@ -11,11 +11,11 @@ define i32 @fptosi_i32_fp128(fp128 %X) nounwind {
; LA32-NEXT: ld.w $a2, $a0, 4
; LA32-NEXT: ld.w $a3, $a0, 8
; LA32-NEXT: ld.w $a0, $a0, 12
-; LA32-NEXT: st.w $a0, $sp, 20
-; LA32-NEXT: st.w $a3, $sp, 16
-; LA32-NEXT: st.w $a2, $sp, 12
-; LA32-NEXT: addi.w $a0, $sp, 8
-; LA32-NEXT: st.w $a1, $sp, 8
+; LA32-NEXT: st.w $a0, $sp, 12
+; LA32-NEXT: st.w $a3, $sp, 8
+; LA32-NEXT: st.w $a2, $sp, 4
+; LA32-NEXT: addi.w $a0, $sp, 0
+; LA32-NEXT: st.w $a1, $sp, 0
; LA32-NEXT: bl __fixtfsi
; LA32-NEXT: ld.w $ra, $sp, 28 # 4-byte Folded Reload
; LA32-NEXT: addi.w $sp, $sp, 32
diff --git a/llvm/test/CodeGen/MSP430/llvm.exp10.ll b/llvm/test/CodeGen/MSP430/llvm.exp10.ll
index 7d4cf7e3fab39..80df1e52c654c 100644
--- a/llvm/test/CodeGen/MSP430/llvm.exp10.ll
+++ b/llvm/test/CodeGen/MSP430/llvm.exp10.ll
@@ -116,16 +116,16 @@ define fp128 @exp10_f128(fp128 %x) #0 {
; CHECK-LABEL: exp10_f128:
; CHECK: ; %bb.0:
; CHECK-NEXT: push r10
-; CHECK-NEXT: sub #32, r1
+; CHECK-NEXT: sub #44, r1
; CHECK-NEXT: mov r12, r10
-; CHECK-NEXT: mov 50(r1), 14(r1)
-; CHECK-NEXT: mov 48(r1), 12(r1)
-; CHECK-NEXT: mov 46(r1), 10(r1)
-; CHECK-NEXT: mov 44(r1), 8(r1)
-; CHECK-NEXT: mov 42(r1), 6(r1)
-; CHECK-NEXT: mov 40(r1), 4(r1)
-; CHECK-NEXT: mov 38(r1), 2(r1)
-; CHECK-NEXT: mov 36(r1), 0(r1)
+; CHECK-NEXT: mov 62(r1), 14(r1)
+; CHECK-NEXT: mov 60(r1), 12(r1)
+; CHECK-NEXT: mov 58(r1), 10(r1)
+; CHECK-NEXT: mov 56(r1), 8(r1)
+; CHECK-NEXT: mov 54(r1), 6(r1)
+; CHECK-NEXT: mov 52(r1), 4(r1)
+; CHECK-NEXT: mov 50(r1), 2(r1)
+; CHECK-NEXT: mov 48(r1), 0(r1)
; CHECK-NEXT: mov r1, r12
; CHECK-NEXT: add #16, r12
; CHECK-NEXT: call #exp10l
@@ -137,7 +137,7 @@ define fp128 @exp10_f128(fp128 %x) #0 {
; CHECK-NEXT: mov 20(r1), 4(r10)
; CHECK-NEXT: mov 18(r1), 2(r10)
; CHECK-NEXT: mov 16(r1), 0(r10)
-; CHECK-NEXT: add #32, r1
+; CHECK-NEXT: add #44, r1
; CHECK-NEXT: pop r10
; CHECK-NEXT: ret
%r = call fp128 @llvm.exp10.f128(fp128 %x)
@@ -148,27 +148,27 @@ define <2 x fp128> @exp10_v2f128(<2 x fp128> %x) #0 {
; CHECK-LABEL: exp10_v2f128:
; CHECK: ; %bb.0:
; CHECK-NEXT: push r10
-; CHECK-NEXT: sub #48, r1
+; CHECK-NEXT: sub #60, r1
; CHECK-NEXT: mov r12, r10
-; CHECK-NEXT: mov 82(r1), 14(r1)
-; CHECK-NEXT: mov 80(r1), 12(r1)
-; CHECK-NEXT: mov 78(r1), 10(r1)
-; CHECK-NEXT: mov 76(r1), 8(r1)
-; CHECK-NEXT: mov 74(r1), 6(r1)
-; CHECK-NEXT: mov 72(r1), 4(r1)
-; CHECK-NEXT: mov 70(r1), 2(r1)
-; CHECK-NEXT: mov 68(r1), 0(r1)
+; CHECK-NEXT: mov 94(r1), 14(r1)
+; CHECK-NEXT: mov 92(r1), 12(r1)
+; CHECK-NEXT: mov 90(r1), 10(r1)
+; CHECK-NEXT: mov 88(r1), 8(r1)
+; CHECK-NEXT: mov 86(r1), 6(r1)
+; CHECK-NEXT: mov 84(r1), 4(r1)
+; CHECK-NEXT: mov 82(r1), 2(r1)
+; CHECK-NEXT: mov 80(r1), 0(r1)
; CHECK-NEXT: mov r1, r12
; CHECK-NEXT: add #32, r12
; CHECK-NEXT: call #exp10l
-; CHECK-NEXT: mov 66(r1), 14(r1)
-; CHECK-NEXT: mov 64(r1), 12(r1)
-; CHECK-NEXT: mov 62(r1), 10(r1)
-; CHECK-NEXT: mov 60(r1), 8(r1)
-; CHECK-NEXT: mov 58(r1), 6(r1)
-; CHECK-NEXT: mov 56(r1), 4(r1)
-; CHECK-NEXT: mov 54(r1), 2(r1)
-; CHECK-NEXT: mov 52(r1), 0(r1)
+; CHECK-NEXT: mov 78(r1), 14(r1)
+; CHECK-NEXT: mov 76(r1), 12(r1)
+; CHECK-NEXT: mov 74(r1), 10(r1)
+; CHECK-NEXT: mov 72(r1), 8(r1)
+; CHECK-NEXT: mov 70(r1), 6(r1)
+; CHECK-NEXT: mov 68(r1), 4(r1)
+; CHECK-NEXT: mov 66(r1), 2(r1)
+; CHECK-NEXT: mov 64(r1), 0(r1)
; CHECK-NEXT: mov r1, r12
; CHECK-NEXT: add #16, r12
; CHECK-NEXT: call #exp10l
@@ -188,7 +188,7 @@ define <2 x fp128> @exp10_v2f128(<2 x fp128> %x) #0 {
; CHECK-NEXT: mov 20(r1), 4(r10)
; CHECK-NEXT: mov 18(r1), 2(r10)
; CHECK-NEXT: mov 16(r1), 0(r10)
-; CHECK-NEXT: add #48, r1
+; CHECK-NEXT: add #60, r1
; CHECK-NEXT: pop r10
; CHECK-NEXT: ret
%r = call <2 x fp128> @llvm.exp10.v2f128(<2 x fp128> %x)
diff --git a/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/add.ll b/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/add.ll
index f76d18b53e59b..aebfdb6607af8 100644
--- a/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/add.ll
+++ b/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/add.ll
@@ -100,15 +100,25 @@ entry:
define i128 @add_i128(i128 %a, i128 %b) {
; MIPS32-LABEL: add_i128:
; MIPS32: # %bb.0: # %entry
+; MIPS32-NEXT: addiu $sp, $sp, -16
+; MIPS32-NEXT: .cfi_def_cfa_offset 16
+; MIPS32-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: .cfi_offset 30, -8
+; MIPS32-NEXT: move $fp, $sp
+; MIPS32-NEXT: .cfi_def_cfa_register 30
+; MIPS32-NEXT: addiu $1, $zero, -16
+; MIPS32-NEXT: and $sp, $sp, $1
; MIPS32-NEXT: move $8, $4
; MIPS32-NEXT: move $3, $5
-; MIPS32-NEXT: addiu $1, $sp, 16
+; MIPS32-NEXT: addiu $1, $fp, 32
; MIPS32-NEXT: lw $2, 0($1)
-; MIPS32-NEXT: addiu $1, $sp, 20
+; MIPS32-NEXT: addiu $1, $fp, 36
; MIPS32-NEXT: lw $4, 0($1)
-; MIPS32-NEXT: addiu $1, $sp, 24
+; MIPS32-NEXT: addiu $1, $fp, 40
; MIPS32-NEXT: lw $5, 0($1)
-; MIPS32-NEXT: addiu $1, $sp, 28
+; MIPS32-NEXT: addiu $1, $fp, 44
; MIPS32-NEXT: lw $1, 0($1)
; MIPS32-NEXT: addu $2, $2, $8
; MIPS32-NEXT: sltu $9, $2, $8
@@ -128,6 +138,10 @@ define i128 @add_i128(i128 %a, i128 %b) {
; MIPS32-NEXT: addu $1, $1, $7
; MIPS32-NEXT: andi $5, $5, 1
; MIPS32-NEXT: addu $5, $1, $5
+; MIPS32-NEXT: move $sp, $fp
+; MIPS32-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: addiu $sp, $sp, 16
; MIPS32-NEXT: jr $ra
; MIPS32-NEXT: nop
entry:
diff --git a/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/mul.ll b/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/mul.ll
index 2ee338c96f9d9..e26cbba39303a 100644
--- a/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/mul.ll
+++ b/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/mul.ll
@@ -104,17 +104,27 @@ entry:
define i128 @mul_i128(i128 %a, i128 %b) {
; MIPS32-LABEL: mul_i128:
; MIPS32: # %bb.0: # %entry
+; MIPS32-NEXT: addiu $sp, $sp, -16
+; MIPS32-NEXT: .cfi_def_cfa_offset 16
+; MIPS32-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: .cfi_offset 30, -8
+; MIPS32-NEXT: move $fp, $sp
+; MIPS32-NEXT: .cfi_def_cfa_register 30
+; MIPS32-NEXT: addiu $1, $zero, -16
+; MIPS32-NEXT: and $sp, $sp, $1
; MIPS32-NEXT: move $12, $4
; MIPS32-NEXT: move $13, $5
; MIPS32-NEXT: move $14, $6
; MIPS32-NEXT: move $15, $7
-; MIPS32-NEXT: addiu $1, $sp, 16
+; MIPS32-NEXT: addiu $1, $fp, 32
; MIPS32-NEXT: lw $9, 0($1)
-; MIPS32-NEXT: addiu $1, $sp, 20
+; MIPS32-NEXT: addiu $1, $fp, 36
; MIPS32-NEXT: lw $8, 0($1)
-; MIPS32-NEXT: addiu $1, $sp, 24
+; MIPS32-NEXT: addiu $1, $fp, 40
; MIPS32-NEXT: lw $7, 0($1)
-; MIPS32-NEXT: addiu $1, $sp, 28
+; MIPS32-NEXT: addiu $1, $fp, 44
; MIPS32-NEXT: lw $6, 0($1)
; MIPS32-NEXT: mul $2, $9, $12
; MIPS32-NEXT: multu $9, $12
@@ -164,6 +174,10 @@ define i128 @mul_i128(i128 %a, i128 %b) {
; MIPS32-NEXT: addu $1, $1, $7
; MIPS32-NEXT: addu $1, $1, $6
; MIPS32-NEXT: addu $5, $1, $5
+; MIPS32-NEXT: move $sp, $fp
+; MIPS32-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: addiu $sp, $sp, 16
; MIPS32-NEXT: jr $ra
; MIPS32-NEXT: nop
entry:
diff --git a/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/sub.ll b/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/sub.ll
index e9ada30d43825..872b6d84eb54f 100644
--- a/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/sub.ll
+++ b/llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/sub.ll
@@ -101,14 +101,24 @@ entry:
define i128 @sub_i128(i128 %a, i128 %b) {
; MIPS32-LABEL: sub_i128:
; MIPS32: # %bb.0: # %entry
+; MIPS32-NEXT: addiu $sp, $sp, -16
+; MIPS32-NEXT: .cfi_def_cfa_offset 16
+; MIPS32-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: .cfi_offset 30, -8
+; MIPS32-NEXT: move $fp, $sp
+; MIPS32-NEXT: .cfi_def_cfa_register 30
+; MIPS32-NEXT: addiu $1, $zero, -16
+; MIPS32-NEXT: and $sp, $sp, $1
; MIPS32-NEXT: move $9, $4
-; MIPS32-NEXT: addiu $1, $sp, 16
+; MIPS32-NEXT: addiu $1, $fp, 32
; MIPS32-NEXT: lw $8, 0($1)
-; MIPS32-NEXT: addiu $1, $sp, 20
+; MIPS32-NEXT: addiu $1, $fp, 36
; MIPS32-NEXT: lw $3, 0($1)
-; MIPS32-NEXT: addiu $1, $sp, 24
+; MIPS32-NEXT: addiu $1, $fp, 40
; MIPS32-NEXT: lw $4, 0($1)
-; MIPS32-NEXT: addiu $1, $sp, 28
+; MIPS32-NEXT: addiu $1, $fp, 44
; MIPS32-NEXT: lw $1, 0($1)
; MIPS32-NEXT: subu $2, $8, $9
; MIPS32-NEXT: sltu $9, $8, $9
@@ -128,6 +138,10 @@ define i128 @sub_i128(i128 %a, i128 %b) {
; MIPS32-NEXT: subu $1, $1, $7
; MIPS32-NEXT: andi $5, $5, 1
; MIPS32-NEXT: subu $5, $1, $5
+; MIPS32-NEXT: move $sp, $fp
+; MIPS32-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: addiu $sp, $sp, 16
; MIPS32-NEXT: jr $ra
; MIPS32-NEXT: nop
entry:
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/add-dsp.ll b/llvm/test/CodeGen/Mips/llvm-ir/add-dsp.ll
index 9f61896829eb9..6f4176c0a3c1a 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/add-dsp.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/add-dsp.ll
@@ -23,50 +23,78 @@ entry:
define i128 @m(i128 zeroext %a, i128 zeroext %b) {
; DSP-LABEL: m:
; DSP: # %bb.0: # %entry
-; DSP-NEXT: lw $1, 28($sp)
-; DSP-NEXT: lw $2, 24($sp)
+; DSP-NEXT: addiu $sp, $sp, -16
+; DSP-NEXT: .cfi_def_cfa_offset 16
+; DSP-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; DSP-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; DSP-NEXT: .cfi_offset 31, -4
+; DSP-NEXT: .cfi_offset 30, -8
+; DSP-NEXT: move $fp, $sp
+; DSP-NEXT: .cfi_def_cfa_register 30
+; DSP-NEXT: addiu $1, $zero, -16
+; DSP-NEXT: and $sp, $sp, $1
+; DSP-NEXT: lw $1, 44($fp)
+; DSP-NEXT: lw $2, 40($fp)
; DSP-NEXT: addsc $1, $1, $7
; DSP-NEXT: addwc $6, $2, $6
; DSP-NEXT: rddsp $2, 1
; DSP-NEXT: ext $3, $2, 20, 1
; DSP-NEXT: ins $3, $2, 6, 1
; DSP-NEXT: ins $3, $zero, 20, 1
-; DSP-NEXT: lw $2, 20($sp)
+; DSP-NEXT: lw $2, 36($fp)
; DSP-NEXT: wrdsp $3, 1
; DSP-NEXT: addwc $3, $2, $5
; DSP-NEXT: rddsp $2, 1
; DSP-NEXT: ext $5, $2, 20, 1
-; DSP-NEXT: lw $7, 16($sp)
+; DSP-NEXT: lw $7, 32($fp)
; DSP-NEXT: ins $5, $2, 6, 1
; DSP-NEXT: ins $5, $zero, 20, 1
; DSP-NEXT: wrdsp $5, 1
; DSP-NEXT: addwc $2, $7, $4
; DSP-NEXT: move $4, $6
-; DSP-NEXT: jr $ra
; DSP-NEXT: move $5, $1
+; DSP-NEXT: move $sp, $fp
+; DSP-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; DSP-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; DSP-NEXT: jr $ra
+; DSP-NEXT: addiu $sp, $sp, 16
;
; MMDSP-LABEL: m:
; MMDSP: # %bb.0: # %entry
-; MMDSP-NEXT: lw $1, 28($sp)
-; MMDSP-NEXT: lw $2, 24($sp)
+; MMDSP-NEXT: addiusp -16
+; MMDSP-NEXT: .cfi_def_cfa_offset 16
+; MMDSP-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MMDSP-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MMDSP-NEXT: .cfi_offset 31, -4
+; MMDSP-NEXT: .cfi_offset 30, -8
+; MMDSP-NEXT: move $fp, $sp
+; MMDSP-NEXT: .cfi_def_cfa_register 30
+; MMDSP-NEXT: addiu $1, $zero, -16
+; MMDSP-NEXT: and $sp, $sp, $1
+; MMDSP-NEXT: lw $1, 44($fp)
+; MMDSP-NEXT: lw $2, 40($fp)
; MMDSP-NEXT: addsc $1, $1, $7
; MMDSP-NEXT: addwc $6, $2, $6
; MMDSP-NEXT: rddsp $2, 1
; MMDSP-NEXT: ext $3, $2, 20, 1
; MMDSP-NEXT: ins $3, $2, 6, 1
; MMDSP-NEXT: ins $3, $zero, 20, 1
-; MMDSP-NEXT: lw $2, 20($sp)
+; MMDSP-NEXT: lw $2, 36($fp)
; MMDSP-NEXT: wrdsp $3, 1
; MMDSP-NEXT: addwc $3, $2, $5
; MMDSP-NEXT: rddsp $2, 1
; MMDSP-NEXT: ext $5, $2, 20, 1
-; MMDSP-NEXT: lw $7, 16($sp)
+; MMDSP-NEXT: lw $7, 32($fp)
; MMDSP-NEXT: ins $5, $2, 6, 1
; MMDSP-NEXT: ins $5, $zero, 20, 1
; MMDSP-NEXT: wrdsp $5, 1
; MMDSP-NEXT: addwc $2, $7, $4
; MMDSP-NEXT: move $4, $6
; MMDSP-NEXT: move $5, $1
+; MMDSP-NEXT: move $sp, $fp
+; MMDSP-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MMDSP-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MMDSP-NEXT: addiusp 16
; MMDSP-NEXT: jrc $ra
entry:
%add = add i128 %b, %a
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/add.ll b/llvm/test/CodeGen/Mips/llvm-ir/add.ll
index a21477acd5341..f78408c140093 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/add.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/add.ll
@@ -123,9 +123,9 @@ entry:
; PRE4: move $[[R1:[0-9]+]], $5
; PRE4: move $[[R2:[0-9]+]], $4
- ; PRE4: lw $[[R3:[0-9]+]], 24($sp)
+ ; PRE4: lw $[[R3:[0-9]+]], 40($fp)
; PRE4: addu $[[R4:[0-9]+]], $6, $[[R3]]
- ; PRE4: lw $[[R5:[0-9]+]], 28($sp)
+ ; PRE4: lw $[[R5:[0-9]+]], 44($fp)
; PRE4: addu $[[R6:[0-9]+]], $7, $[[R5]]
; PRE4: sltu $[[R7:[0-9]+]], $[[R6]], $7
; PRE4: addu $[[R8:[0-9]+]], $[[R4]], $[[R7]]
@@ -133,9 +133,9 @@ entry:
; PRE4: sltiu $[[R10:[0-9]+]], $[[R9]], 1
; PRE4: bnez $[[R10]], $BB5_2
; PRE4: sltu $[[R7]], $[[R8]], $6
- ; PRE4: lw $[[R12:[0-9]+]], 20($sp)
+ ; PRE4: lw $[[R12:[0-9]+]], 36($fp)
; PRE4: addu $[[R13:[0-9]+]], $[[R1]], $[[R12]]
- ; PRE4: lw $[[R14:[0-9]+]], 16($sp)
+ ; PRE4: lw $[[R14:[0-9]+]], 32($fp)
; PRE4: addu $[[R15:[0-9]+]], $[[R13]], $[[R7]]
; PRE4: addu $[[R16:[0-9]+]], $[[R2]], $[[R14]]
; PRE4: sltu $[[R17:[0-9]+]], $[[R15]], $[[R13]]
@@ -143,19 +143,19 @@ entry:
; PRE4: addu $[[R19:[0-9]+]], $[[R16]], $[[R18]]
; PRE4: addu $2, $[[R19]], $[[R17]]
- ; GP32-CMOV: lw $[[T0:[0-9]+]], 24($sp)
+ ; GP32-CMOV: lw $[[T0:[0-9]+]], 40($fp)
; GP32-CMOV: addu $[[T1:[0-9]+]], $6, $[[T0]]
- ; GP32-CMOV: lw $[[T2:[0-9]+]], 28($sp)
+ ; GP32-CMOV: lw $[[T2:[0-9]+]], 44($fp)
; GP32-CMOV: addu $[[T3:[0-9]+]], $7, $[[T2]]
; GP32-CMOV: sltu $[[T4:[0-9]+]], $[[T3]], $7
; GP32-CMOV: addu $[[T5:[0-9]+]], $[[T1]], $[[T4]]
; GP32-CMOV: sltu $[[T6:[0-9]+]], $[[T5]], $6
; GP32-CMOV: xor $[[T7:[0-9]+]], $[[T5]], $6
- ; GP32-CMOV: movz $[[T8:[0-9]+]], $[[T4]], $[[T7]]
- ; GP32-CMOV: lw $[[T9:[0-9]+]], 20($sp)
- ; GP32-CMOV: addu $[[T10:[0-9]+]], $5, $[[T4]]
- ; GP32-CMOV: addu $[[T11:[0-9]+]], $[[T10]], $[[T8]]
- ; GP32-CMOV: lw $[[T12:[0-9]+]], 16($sp)
+ ; GP32-CMOV: movz $[[T6]], $[[T4]], $[[T7]]
+ ; GP32-CMOV: lw $[[T9:[0-9]+]], 36($fp)
+ ; GP32-CMOV: addu $[[T10:[0-9]+]], $5, $[[T9]]
+ ; GP32-CMOV: addu $[[T11:[0-9]+]], $[[T10]], $[[T6]]
+ ; GP32-CMOV: lw $[[T12:[0-9]+]], 32($fp)
; GP32-CMOV: sltu $[[T13:[0-9]+]], $[[T11]], $[[T10]]
; GP32-CMOV: addu $[[T14:[0-9]+]], $4, $[[T12]]
; GP32-CMOV: sltu $[[T15:[0-9]+]], $[[T10]], $5
@@ -174,48 +174,48 @@ entry:
; GP64: daddu $2, $[[T0]], $[[T4]]
; MMR3: move $[[T1:[0-9]+]], $5
- ; MMR3-DAG: lw $[[T2:[0-9]+]], 32($sp)
+ ; MMR3: lw $[[T2:[0-9]+]], 40($fp)
; MMR3: addu16 $[[T3:[0-9]+]], $6, $[[T2]]
- ; MMR3-DAG: lw $[[T4:[0-9]+]], 36($sp)
+ ; MMR3: lw $[[T4:[0-9]+]], 44($fp)
; MMR3: addu16 $[[T5:[0-9]+]], $7, $[[T4]]
; MMR3: sltu $[[T6:[0-9]+]], $[[T5]], $7
; MMR3: addu16 $[[T7:[0-9]+]], $[[T3]], $[[T6]]
; MMR3: sltu $[[T8:[0-9]+]], $[[T7]], $6
; MMR3: xor $[[T9:[0-9]+]], $[[T7]], $6
; MMR3: movz $[[T8]], $[[T6]], $[[T9]]
- ; MMR3: lw $[[T10:[0-9]+]], 28($sp)
+ ; MMR3: lw $[[T10:[0-9]+]], 36($fp)
; MMR3: addu16 $[[T11:[0-9]+]], $[[T1]], $[[T10]]
; MMR3: addu16 $[[T12:[0-9]+]], $[[T11]], $[[T8]]
- ; MMR3: lw $[[T13:[0-9]+]], 24($sp)
+ ; MMR3: lw $[[T13:[0-9]+]], 32($fp)
; MMR3: sltu $[[T14:[0-9]+]], $[[T12]], $[[T11]]
; MMR3: addu16 $[[T15:[0-9]+]], $4, $[[T13]]
; MMR3: sltu $[[T16:[0-9]+]], $[[T11]], $[[T1]]
; MMR3: addu16 $[[T17:[0-9]+]], $[[T15]], $[[T16]]
- ; MMR3: addu16 $2, $2, $[[T14]]
+ ; MMR3: addu16 $2, $[[T17]], $[[T14]]
; MMR6: move $[[T1:[0-9]+]], $5
; MMR6: move $[[T2:[0-9]+]], $4
- ; MMR6: lw $[[T3:[0-9]+]], 32($sp)
+ ; MMR6: lw $[[T3:[0-9]+]], 40($fp)
; MMR6: addu16 $[[T4:[0-9]+]], $6, $[[T3]]
- ; MMR6: lw $[[T5:[0-9]+]], 36($sp)
+ ; MMR6: lw $[[T5:[0-9]+]], 44($fp)
; MMR6: addu16 $[[T6:[0-9]+]], $7, $[[T5]]
; MMR6: sltu $[[T7:[0-9]+]], $[[T6]], $7
- ; MMR6: addu16 $[[T8:[0-9]+]], $[[T4]], $7
+ ; MMR6: addu16 $[[T8:[0-9]+]], $[[T4]], $[[T7]]
; MMR6: sltu $[[T9:[0-9]+]], $[[T8]], $6
- ; MMR6: xor $[[T10:[0-9]+]], $[[T4]], $6
+ ; MMR6: xor $[[T10:[0-9]+]], $[[T8]], $6
; MMR6: sltiu $[[T11:[0-9]+]], $[[T10]], 1
; MMR6: seleqz $[[T12:[0-9]+]], $[[T9]], $[[T11]]
; MMR6: selnez $[[T13:[0-9]+]], $[[T7]], $[[T11]]
- ; MMR6: lw $[[T14:[0-9]+]], 24($sp)
+ ; MMR6: lw $[[T14:[0-9]+]], 32($fp)
; MMR6: or $[[T15:[0-9]+]], $[[T13]], $[[T12]]
; MMR6: addu16 $[[T16:[0-9]+]], $[[T2]], $[[T14]]
- ; MMR6: lw $[[T17:[0-9]+]], 28($sp)
+ ; MMR6: lw $[[T17:[0-9]+]], 36($fp)
; MMR6: addu16 $[[T18:[0-9]+]], $[[T1]], $[[T17]]
; MMR6: addu16 $[[T19:[0-9]+]], $[[T18]], $[[T15]]
; MMR6: sltu $[[T20:[0-9]+]], $[[T18]], $[[T1]]
- ; MMR6: sltu $[[T21:[0-9]+]], $[[T17]], $[[T18]]
- ; MMR6: addu16 $2, $[[T16]], $[[T20]]
- ; MMR6: addu16 $2, $[[T20]], $[[T21]]
+ ; MMR6: sltu $[[T21:[0-9]+]], $[[T19]], $[[T18]]
+ ; MMR6: addu16 $[[T22:[0-9]+]], $[[T16]], $[[T20]]
+ ; MMR6: addu16 $2, $[[T22]], $[[T21]]
%r = add i128 %a, %b
ret i128 %r
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/and.ll b/llvm/test/CodeGen/Mips/llvm-ir/and.ll
index 87f7edb2b7132..c7c77b732699f 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/and.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/and.ll
@@ -278,39 +278,81 @@ entry:
define signext i128 @and_i128(i128 signext %a, i128 signext %b) {
; MIPS-LABEL: and_i128:
; MIPS: # %bb.0: # %entry
-; MIPS-NEXT: lw $1, 20($sp)
-; MIPS-NEXT: lw $2, 16($sp)
+; MIPS-NEXT: addiu $sp, $sp, -16
+; MIPS-NEXT: .cfi_def_cfa_offset 16
+; MIPS-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MIPS-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MIPS-NEXT: .cfi_offset 31, -4
+; MIPS-NEXT: .cfi_offset 30, -8
+; MIPS-NEXT: move $fp, $sp
+; MIPS-NEXT: .cfi_def_cfa_register 30
+; MIPS-NEXT: addiu $1, $zero, -16
+; MIPS-NEXT: and $sp, $sp, $1
+; MIPS-NEXT: lw $1, 36($fp)
+; MIPS-NEXT: lw $2, 32($fp)
; MIPS-NEXT: and $2, $4, $2
; MIPS-NEXT: and $3, $5, $1
-; MIPS-NEXT: lw $1, 24($sp)
+; MIPS-NEXT: lw $1, 40($fp)
; MIPS-NEXT: and $4, $6, $1
-; MIPS-NEXT: lw $1, 28($sp)
-; MIPS-NEXT: jr $ra
+; MIPS-NEXT: lw $1, 44($fp)
; MIPS-NEXT: and $5, $7, $1
+; MIPS-NEXT: move $sp, $fp
+; MIPS-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MIPS-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MIPS-NEXT: jr $ra
+; MIPS-NEXT: addiu $sp, $sp, 16
;
; MIPS32R2-LABEL: and_i128:
; MIPS32R2: # %bb.0: # %entry
-; MIPS32R2-NEXT: lw $1, 20($sp)
-; MIPS32R2-NEXT: lw $2, 16($sp)
+; MIPS32R2-NEXT: addiu $sp, $sp, -16
+; MIPS32R2-NEXT: .cfi_def_cfa_offset 16
+; MIPS32R2-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MIPS32R2-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MIPS32R2-NEXT: .cfi_offset 31, -4
+; MIPS32R2-NEXT: .cfi_offset 30, -8
+; MIPS32R2-NEXT: move $fp, $sp
+; MIPS32R2-NEXT: .cfi_def_cfa_register 30
+; MIPS32R2-NEXT: addiu $1, $zero, -16
+; MIPS32R2-NEXT: and $sp, $sp, $1
+; MIPS32R2-NEXT: lw $1, 36($fp)
+; MIPS32R2-NEXT: lw $2, 32($fp)
; MIPS32R2-NEXT: and $2, $4, $2
; MIPS32R2-NEXT: and $3, $5, $1
-; MIPS32R2-NEXT: lw $1, 24($sp)
+; MIPS32R2-NEXT: lw $1, 40($fp)
; MIPS32R2-NEXT: and $4, $6, $1
-; MIPS32R2-NEXT: lw $1, 28($sp)
-; MIPS32R2-NEXT: jr $ra
+; MIPS32R2-NEXT: lw $1, 44($fp)
; MIPS32R2-NEXT: and $5, $7, $1
+; MIPS32R2-NEXT: move $sp, $fp
+; MIPS32R2-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MIPS32R2-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MIPS32R2-NEXT: jr $ra
+; MIPS32R2-NEXT: addiu $sp, $sp, 16
;
; MIPS32R6-LABEL: and_i128:
; MIPS32R6: # %bb.0: # %entry
-; MIPS32R6-NEXT: lw $1, 20($sp)
-; MIPS32R6-NEXT: lw $2, 16($sp)
+; MIPS32R6-NEXT: addiu $sp, $sp, -16
+; MIPS32R6-NEXT: .cfi_def_cfa_offset 16
+; MIPS32R6-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MIPS32R6-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MIPS32R6-NEXT: .cfi_offset 31, -4
+; MIPS32R6-NEXT: .cfi_offset 30, -8
+; MIPS32R6-NEXT: move $fp, $sp
+; MIPS32R6-NEXT: .cfi_def_cfa_register 30
+; MIPS32R6-NEXT: addiu $1, $zero, -16
+; MIPS32R6-NEXT: and $sp, $sp, $1
+; MIPS32R6-NEXT: lw $1, 36($fp)
+; MIPS32R6-NEXT: lw $2, 32($fp)
; MIPS32R6-NEXT: and $2, $4, $2
; MIPS32R6-NEXT: and $3, $5, $1
-; MIPS32R6-NEXT: lw $1, 24($sp)
+; MIPS32R6-NEXT: lw $1, 40($fp)
; MIPS32R6-NEXT: and $4, $6, $1
-; MIPS32R6-NEXT: lw $1, 28($sp)
-; MIPS32R6-NEXT: jr $ra
+; MIPS32R6-NEXT: lw $1, 44($fp)
; MIPS32R6-NEXT: and $5, $7, $1
+; MIPS32R6-NEXT: move $sp, $fp
+; MIPS32R6-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MIPS32R6-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MIPS32R6-NEXT: jr $ra
+; MIPS32R6-NEXT: addiu $sp, $sp, 16
;
; MIPS64-LABEL: and_i128:
; MIPS64: # %bb.0: # %entry
@@ -332,25 +374,53 @@ define signext i128 @and_i128(i128 signext %a, i128 signext %b) {
;
; MM32R3-LABEL: and_i128:
; MM32R3: # %bb.0: # %entry
-; MM32R3-NEXT: lwp $2, 16($sp)
+; MM32R3-NEXT: addiusp -16
+; MM32R3-NEXT: .cfi_def_cfa_offset 16
+; MM32R3-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MM32R3-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MM32R3-NEXT: .cfi_offset 31, -4
+; MM32R3-NEXT: .cfi_offset 30, -8
+; MM32R3-NEXT: move $fp, $sp
+; MM32R3-NEXT: .cfi_def_cfa_register 30
+; MM32R3-NEXT: addiu $1, $zero, -16
+; MM32R3-NEXT: and $sp, $sp, $1
+; MM32R3-NEXT: lwp $2, 32($fp)
; MM32R3-NEXT: and16 $2, $4
; MM32R3-NEXT: and16 $3, $5
-; MM32R3-NEXT: lw $4, 24($sp)
+; MM32R3-NEXT: lw $4, 40($fp)
; MM32R3-NEXT: and16 $4, $6
-; MM32R3-NEXT: lw $5, 28($sp)
+; MM32R3-NEXT: lw $5, 44($fp)
; MM32R3-NEXT: and16 $5, $7
+; MM32R3-NEXT: move $sp, $fp
+; MM32R3-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MM32R3-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MM32R3-NEXT: addiusp 16
; MM32R3-NEXT: jrc $ra
;
; MM32R6-LABEL: and_i128:
; MM32R6: # %bb.0: # %entry
-; MM32R6-NEXT: lw $1, 20($sp)
-; MM32R6-NEXT: lw $2, 16($sp)
+; MM32R6-NEXT: addiu $sp, $sp, -16
+; MM32R6-NEXT: .cfi_def_cfa_offset 16
+; MM32R6-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MM32R6-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MM32R6-NEXT: .cfi_offset 31, -4
+; MM32R6-NEXT: .cfi_offset 30, -8
+; MM32R6-NEXT: move $fp, $sp
+; MM32R6-NEXT: .cfi_def_cfa_register 30
+; MM32R6-NEXT: addiu $1, $zero, -16
+; MM32R6-NEXT: and $sp, $sp, $1
+; MM32R6-NEXT: lw $1, 36($fp)
+; MM32R6-NEXT: lw $2, 32($fp)
; MM32R6-NEXT: and $2, $4, $2
; MM32R6-NEXT: and $3, $5, $1
-; MM32R6-NEXT: lw $1, 24($sp)
+; MM32R6-NEXT: lw $1, 40($fp)
; MM32R6-NEXT: and $4, $6, $1
-; MM32R6-NEXT: lw $1, 28($sp)
+; MM32R6-NEXT: lw $1, 44($fp)
; MM32R6-NEXT: and $5, $7, $1
+; MM32R6-NEXT: move $sp, $fp
+; MM32R6-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MM32R6-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MM32R6-NEXT: addiu $sp, $sp, 16
; MM32R6-NEXT: jrc $ra
entry:
%r = and i128 %a, %b
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/ashr.ll b/llvm/test/CodeGen/Mips/llvm-ir/ashr.ll
index 2b8129acb91fc..ecafcd012ff61 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/ashr.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/ashr.ll
@@ -380,8 +380,16 @@ entry:
define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; MIPS-LABEL: ashr_i128:
; MIPS: # %bb.0: # %entry
-; MIPS-NEXT: addiu $sp, $sp, -32
-; MIPS-NEXT: .cfi_def_cfa_offset 32
+; MIPS-NEXT: addiu $sp, $sp, -48
+; MIPS-NEXT: .cfi_def_cfa_offset 48
+; MIPS-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS-NEXT: .cfi_offset 31, -4
+; MIPS-NEXT: .cfi_offset 30, -8
+; MIPS-NEXT: move $fp, $sp
+; MIPS-NEXT: .cfi_def_cfa_register 30
+; MIPS-NEXT: addiu $1, $zero, -16
+; MIPS-NEXT: and $sp, $sp, $1
; MIPS-NEXT: sra $1, $4, 31
; MIPS-NEXT: sw $7, 28($sp)
; MIPS-NEXT: sw $6, 24($sp)
@@ -393,7 +401,7 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; MIPS-NEXT: sw $1, 0($sp)
; MIPS-NEXT: addiu $1, $sp, 0
; MIPS-NEXT: addiu $1, $1, 16
-; MIPS-NEXT: lw $2, 60($sp)
+; MIPS-NEXT: lw $2, 76($fp)
; MIPS-NEXT: srl $3, $2, 3
; MIPS-NEXT: andi $3, $3, 12
; MIPS-NEXT: subu $1, $1, $3
@@ -416,13 +424,24 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; MIPS-NEXT: sllv $5, $5, $7
; MIPS-NEXT: or $5, $1, $5
; MIPS-NEXT: srav $2, $8, $2
+; MIPS-NEXT: move $sp, $fp
+; MIPS-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS-NEXT: jr $ra
-; MIPS-NEXT: addiu $sp, $sp, 32
+; MIPS-NEXT: addiu $sp, $sp, 48
;
; MIPS32-LABEL: ashr_i128:
; MIPS32: # %bb.0: # %entry
-; MIPS32-NEXT: addiu $sp, $sp, -32
-; MIPS32-NEXT: .cfi_def_cfa_offset 32
+; MIPS32-NEXT: addiu $sp, $sp, -48
+; MIPS32-NEXT: .cfi_def_cfa_offset 48
+; MIPS32-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: .cfi_offset 30, -8
+; MIPS32-NEXT: move $fp, $sp
+; MIPS32-NEXT: .cfi_def_cfa_register 30
+; MIPS32-NEXT: addiu $1, $zero, -16
+; MIPS32-NEXT: and $sp, $sp, $1
; MIPS32-NEXT: sra $1, $4, 31
; MIPS32-NEXT: sw $7, 28($sp)
; MIPS32-NEXT: sw $6, 24($sp)
@@ -434,7 +453,7 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; MIPS32-NEXT: sw $1, 0($sp)
; MIPS32-NEXT: addiu $1, $sp, 0
; MIPS32-NEXT: addiu $1, $1, 16
-; MIPS32-NEXT: lw $2, 60($sp)
+; MIPS32-NEXT: lw $2, 76($fp)
; MIPS32-NEXT: srl $3, $2, 3
; MIPS32-NEXT: andi $3, $3, 12
; MIPS32-NEXT: subu $1, $1, $3
@@ -457,13 +476,24 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; MIPS32-NEXT: sllv $5, $5, $7
; MIPS32-NEXT: or $5, $1, $5
; MIPS32-NEXT: srav $2, $8, $2
+; MIPS32-NEXT: move $sp, $fp
+; MIPS32-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS32-NEXT: jr $ra
-; MIPS32-NEXT: addiu $sp, $sp, 32
+; MIPS32-NEXT: addiu $sp, $sp, 48
;
; 32R2-LABEL: ashr_i128:
; 32R2: # %bb.0: # %entry
-; 32R2-NEXT: addiu $sp, $sp, -32
-; 32R2-NEXT: .cfi_def_cfa_offset 32
+; 32R2-NEXT: addiu $sp, $sp, -48
+; 32R2-NEXT: .cfi_def_cfa_offset 48
+; 32R2-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; 32R2-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; 32R2-NEXT: .cfi_offset 31, -4
+; 32R2-NEXT: .cfi_offset 30, -8
+; 32R2-NEXT: move $fp, $sp
+; 32R2-NEXT: .cfi_def_cfa_register 30
+; 32R2-NEXT: addiu $1, $zero, -16
+; 32R2-NEXT: and $sp, $sp, $1
; 32R2-NEXT: sra $1, $4, 31
; 32R2-NEXT: sw $7, 28($sp)
; 32R2-NEXT: sw $6, 24($sp)
@@ -475,7 +505,7 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; 32R2-NEXT: sw $1, 0($sp)
; 32R2-NEXT: addiu $1, $sp, 0
; 32R2-NEXT: addiu $1, $1, 16
-; 32R2-NEXT: lw $2, 60($sp)
+; 32R2-NEXT: lw $2, 76($fp)
; 32R2-NEXT: srl $3, $2, 3
; 32R2-NEXT: andi $3, $3, 12
; 32R2-NEXT: subu $1, $1, $3
@@ -498,13 +528,24 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; 32R2-NEXT: sllv $5, $5, $7
; 32R2-NEXT: or $5, $1, $5
; 32R2-NEXT: srav $2, $8, $2
+; 32R2-NEXT: move $sp, $fp
+; 32R2-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; 32R2-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; 32R2-NEXT: jr $ra
-; 32R2-NEXT: addiu $sp, $sp, 32
+; 32R2-NEXT: addiu $sp, $sp, 48
;
; 32R6-LABEL: ashr_i128:
; 32R6: # %bb.0: # %entry
-; 32R6-NEXT: addiu $sp, $sp, -32
-; 32R6-NEXT: .cfi_def_cfa_offset 32
+; 32R6-NEXT: addiu $sp, $sp, -48
+; 32R6-NEXT: .cfi_def_cfa_offset 48
+; 32R6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; 32R6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; 32R6-NEXT: .cfi_offset 31, -4
+; 32R6-NEXT: .cfi_offset 30, -8
+; 32R6-NEXT: move $fp, $sp
+; 32R6-NEXT: .cfi_def_cfa_register 30
+; 32R6-NEXT: addiu $1, $zero, -16
+; 32R6-NEXT: and $sp, $sp, $1
; 32R6-NEXT: sra $1, $4, 31
; 32R6-NEXT: sw $7, 28($sp)
; 32R6-NEXT: sw $6, 24($sp)
@@ -516,7 +557,7 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; 32R6-NEXT: sw $1, 0($sp)
; 32R6-NEXT: addiu $1, $sp, 0
; 32R6-NEXT: addiu $1, $1, 16
-; 32R6-NEXT: lw $2, 60($sp)
+; 32R6-NEXT: lw $2, 76($fp)
; 32R6-NEXT: srl $3, $2, 3
; 32R6-NEXT: andi $3, $3, 12
; 32R6-NEXT: subu $1, $1, $3
@@ -539,8 +580,11 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; 32R6-NEXT: sllv $5, $5, $7
; 32R6-NEXT: or $5, $1, $5
; 32R6-NEXT: srav $2, $8, $2
+; 32R6-NEXT: move $sp, $fp
+; 32R6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; 32R6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; 32R6-NEXT: jr $ra
-; 32R6-NEXT: addiu $sp, $sp, 32
+; 32R6-NEXT: addiu $sp, $sp, 48
;
; MIPS3-LABEL: ashr_i128:
; MIPS3: # %bb.0: # %entry
@@ -613,11 +657,19 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
;
; MMR3-LABEL: ashr_i128:
; MMR3: # %bb.0: # %entry
-; MMR3-NEXT: addiusp -40
-; MMR3-NEXT: .cfi_def_cfa_offset 40
+; MMR3-NEXT: addiusp -48
+; MMR3-NEXT: .cfi_def_cfa_offset 48
+; MMR3-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; MMR3-NEXT: swp $16, 32($sp)
-; MMR3-NEXT: .cfi_offset 17, -4
-; MMR3-NEXT: .cfi_offset 16, -8
+; MMR3-NEXT: .cfi_offset 31, -4
+; MMR3-NEXT: .cfi_offset 30, -8
+; MMR3-NEXT: .cfi_offset 17, -12
+; MMR3-NEXT: .cfi_offset 16, -16
+; MMR3-NEXT: move $fp, $sp
+; MMR3-NEXT: .cfi_def_cfa_register 30
+; MMR3-NEXT: addiu $1, $zero, -16
+; MMR3-NEXT: and $sp, $sp, $1
; MMR3-NEXT: sra $1, $4, 31
; MMR3-NEXT: swp $6, 24($sp)
; MMR3-NEXT: swp $4, 16($sp)
@@ -627,7 +679,7 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; MMR3-NEXT: sw $1, 0($sp)
; MMR3-NEXT: addiur1sp $2, 0
; MMR3-NEXT: addiur2 $2, $2, 16
-; MMR3-NEXT: lw $3, 68($sp)
+; MMR3-NEXT: lw $3, 76($fp)
; MMR3-NEXT: srl16 $4, $3, 3
; MMR3-NEXT: andi $4, $4, 12
; MMR3-NEXT: subu16 $5, $2, $4
@@ -649,16 +701,27 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; MMR3-NEXT: sllv $5, $5, $1
; MMR3-NEXT: or16 $5, $6
; MMR3-NEXT: srav $2, $17, $2
+; MMR3-NEXT: move $sp, $fp
; MMR3-NEXT: lwp $16, 32($sp)
-; MMR3-NEXT: addiusp 40
+; MMR3-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR3-NEXT: addiusp 48
; MMR3-NEXT: jrc $ra
;
; MMR6-LABEL: ashr_i128:
; MMR6: # %bb.0: # %entry
-; MMR6-NEXT: addiu $sp, $sp, -40
-; MMR6-NEXT: .cfi_def_cfa_offset 40
+; MMR6-NEXT: addiu $sp, $sp, -48
+; MMR6-NEXT: .cfi_def_cfa_offset 48
+; MMR6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; MMR6-NEXT: sw $16, 36($sp) # 4-byte Folded Spill
-; MMR6-NEXT: .cfi_offset 16, -4
+; MMR6-NEXT: .cfi_offset 31, -4
+; MMR6-NEXT: .cfi_offset 30, -8
+; MMR6-NEXT: .cfi_offset 16, -12
+; MMR6-NEXT: move $fp, $sp
+; MMR6-NEXT: .cfi_def_cfa_register 30
+; MMR6-NEXT: addiu $1, $zero, -16
+; MMR6-NEXT: and $sp, $sp, $1
; MMR6-NEXT: sra $1, $4, 31
; MMR6-NEXT: sw $7, 28($sp)
; MMR6-NEXT: sw $6, 24($sp)
@@ -670,7 +733,7 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; MMR6-NEXT: sw $1, 0($sp)
; MMR6-NEXT: addiu $2, $sp, 0
; MMR6-NEXT: addiur2 $2, $2, 16
-; MMR6-NEXT: lw $3, 68($sp)
+; MMR6-NEXT: lw $3, 76($fp)
; MMR6-NEXT: srl16 $4, $3, 3
; MMR6-NEXT: andi $4, $4, 12
; MMR6-NEXT: subu16 $2, $2, $4
@@ -693,8 +756,11 @@ define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
; MMR6-NEXT: sllv $2, $2, $7
; MMR6-NEXT: or $5, $1, $2
; MMR6-NEXT: srav $2, $16, $6
+; MMR6-NEXT: move $sp, $fp
; MMR6-NEXT: lw $16, 36($sp) # 4-byte Folded Reload
-; MMR6-NEXT: addiu $sp, $sp, 40
+; MMR6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR6-NEXT: addiu $sp, $sp, 48
; MMR6-NEXT: jrc $ra
entry:
%r = ashr i128 %a, %b
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/lshr.ll b/llvm/test/CodeGen/Mips/llvm-ir/lshr.ll
index 69b842c73db1b..3371eaecdbef5 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/lshr.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/lshr.ll
@@ -396,15 +396,23 @@ entry:
define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
; MIPS2-LABEL: lshr_i128:
; MIPS2: # %bb.0: # %entry
-; MIPS2-NEXT: addiu $sp, $sp, -32
-; MIPS2-NEXT: .cfi_def_cfa_offset 32
+; MIPS2-NEXT: addiu $sp, $sp, -48
+; MIPS2-NEXT: .cfi_def_cfa_offset 48
+; MIPS2-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: .cfi_offset 31, -4
+; MIPS2-NEXT: .cfi_offset 30, -8
+; MIPS2-NEXT: move $fp, $sp
+; MIPS2-NEXT: .cfi_def_cfa_register 30
+; MIPS2-NEXT: addiu $1, $zero, -16
+; MIPS2-NEXT: and $sp, $sp, $1
; MIPS2-NEXT: addiu $1, $sp, 0
; MIPS2-NEXT: sw $7, 28($sp)
; MIPS2-NEXT: sw $6, 24($sp)
; MIPS2-NEXT: sw $5, 20($sp)
; MIPS2-NEXT: sw $4, 16($sp)
; MIPS2-NEXT: addiu $1, $1, 16
-; MIPS2-NEXT: lw $2, 60($sp)
+; MIPS2-NEXT: lw $2, 76($fp)
; MIPS2-NEXT: srl $3, $2, 3
; MIPS2-NEXT: andi $3, $3, 12
; MIPS2-NEXT: subu $1, $1, $3
@@ -431,20 +439,31 @@ define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
; MIPS2-NEXT: sllv $5, $5, $7
; MIPS2-NEXT: or $5, $1, $5
; MIPS2-NEXT: srlv $2, $8, $2
+; MIPS2-NEXT: move $sp, $fp
+; MIPS2-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS2-NEXT: jr $ra
-; MIPS2-NEXT: addiu $sp, $sp, 32
+; MIPS2-NEXT: addiu $sp, $sp, 48
;
; MIPS32-LABEL: lshr_i128:
; MIPS32: # %bb.0: # %entry
-; MIPS32-NEXT: addiu $sp, $sp, -32
-; MIPS32-NEXT: .cfi_def_cfa_offset 32
+; MIPS32-NEXT: addiu $sp, $sp, -48
+; MIPS32-NEXT: .cfi_def_cfa_offset 48
+; MIPS32-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: .cfi_offset 30, -8
+; MIPS32-NEXT: move $fp, $sp
+; MIPS32-NEXT: .cfi_def_cfa_register 30
+; MIPS32-NEXT: addiu $1, $zero, -16
+; MIPS32-NEXT: and $sp, $sp, $1
; MIPS32-NEXT: addiu $1, $sp, 0
; MIPS32-NEXT: sw $7, 28($sp)
; MIPS32-NEXT: sw $6, 24($sp)
; MIPS32-NEXT: sw $5, 20($sp)
; MIPS32-NEXT: sw $4, 16($sp)
; MIPS32-NEXT: addiu $1, $1, 16
-; MIPS32-NEXT: lw $2, 60($sp)
+; MIPS32-NEXT: lw $2, 76($fp)
; MIPS32-NEXT: srl $3, $2, 3
; MIPS32-NEXT: andi $3, $3, 12
; MIPS32-NEXT: subu $1, $1, $3
@@ -471,20 +490,31 @@ define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
; MIPS32-NEXT: sllv $5, $5, $7
; MIPS32-NEXT: or $5, $1, $5
; MIPS32-NEXT: srlv $2, $8, $2
+; MIPS32-NEXT: move $sp, $fp
+; MIPS32-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS32-NEXT: jr $ra
-; MIPS32-NEXT: addiu $sp, $sp, 32
+; MIPS32-NEXT: addiu $sp, $sp, 48
;
; MIPS32R2-LABEL: lshr_i128:
; MIPS32R2: # %bb.0: # %entry
-; MIPS32R2-NEXT: addiu $sp, $sp, -32
-; MIPS32R2-NEXT: .cfi_def_cfa_offset 32
+; MIPS32R2-NEXT: addiu $sp, $sp, -48
+; MIPS32R2-NEXT: .cfi_def_cfa_offset 48
+; MIPS32R2-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS32R2-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS32R2-NEXT: .cfi_offset 31, -4
+; MIPS32R2-NEXT: .cfi_offset 30, -8
+; MIPS32R2-NEXT: move $fp, $sp
+; MIPS32R2-NEXT: .cfi_def_cfa_register 30
+; MIPS32R2-NEXT: addiu $1, $zero, -16
+; MIPS32R2-NEXT: and $sp, $sp, $1
; MIPS32R2-NEXT: addiu $1, $sp, 0
; MIPS32R2-NEXT: sw $7, 28($sp)
; MIPS32R2-NEXT: sw $6, 24($sp)
; MIPS32R2-NEXT: sw $5, 20($sp)
; MIPS32R2-NEXT: sw $4, 16($sp)
; MIPS32R2-NEXT: addiu $1, $1, 16
-; MIPS32R2-NEXT: lw $2, 60($sp)
+; MIPS32R2-NEXT: lw $2, 76($fp)
; MIPS32R2-NEXT: srl $3, $2, 3
; MIPS32R2-NEXT: andi $3, $3, 12
; MIPS32R2-NEXT: subu $1, $1, $3
@@ -511,20 +541,31 @@ define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
; MIPS32R2-NEXT: sllv $5, $5, $7
; MIPS32R2-NEXT: or $5, $1, $5
; MIPS32R2-NEXT: srlv $2, $8, $2
+; MIPS32R2-NEXT: move $sp, $fp
+; MIPS32R2-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS32R2-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS32R2-NEXT: jr $ra
-; MIPS32R2-NEXT: addiu $sp, $sp, 32
+; MIPS32R2-NEXT: addiu $sp, $sp, 48
;
; MIPS32R6-LABEL: lshr_i128:
; MIPS32R6: # %bb.0: # %entry
-; MIPS32R6-NEXT: addiu $sp, $sp, -32
-; MIPS32R6-NEXT: .cfi_def_cfa_offset 32
+; MIPS32R6-NEXT: addiu $sp, $sp, -48
+; MIPS32R6-NEXT: .cfi_def_cfa_offset 48
+; MIPS32R6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS32R6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS32R6-NEXT: .cfi_offset 31, -4
+; MIPS32R6-NEXT: .cfi_offset 30, -8
+; MIPS32R6-NEXT: move $fp, $sp
+; MIPS32R6-NEXT: .cfi_def_cfa_register 30
+; MIPS32R6-NEXT: addiu $1, $zero, -16
+; MIPS32R6-NEXT: and $sp, $sp, $1
; MIPS32R6-NEXT: addiu $1, $sp, 0
; MIPS32R6-NEXT: sw $7, 28($sp)
; MIPS32R6-NEXT: sw $6, 24($sp)
; MIPS32R6-NEXT: sw $5, 20($sp)
; MIPS32R6-NEXT: sw $4, 16($sp)
; MIPS32R6-NEXT: addiu $1, $1, 16
-; MIPS32R6-NEXT: lw $2, 60($sp)
+; MIPS32R6-NEXT: lw $2, 76($fp)
; MIPS32R6-NEXT: srl $3, $2, 3
; MIPS32R6-NEXT: andi $3, $3, 12
; MIPS32R6-NEXT: subu $1, $1, $3
@@ -551,8 +592,11 @@ define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
; MIPS32R6-NEXT: sllv $5, $5, $7
; MIPS32R6-NEXT: or $5, $1, $5
; MIPS32R6-NEXT: srlv $2, $8, $2
+; MIPS32R6-NEXT: move $sp, $fp
+; MIPS32R6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS32R6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS32R6-NEXT: jr $ra
-; MIPS32R6-NEXT: addiu $sp, $sp, 32
+; MIPS32R6-NEXT: addiu $sp, $sp, 48
;
; MIPS3-LABEL: lshr_i128:
; MIPS3: # %bb.0: # %entry
@@ -634,11 +678,19 @@ define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
;
; MMR3-LABEL: lshr_i128:
; MMR3: # %bb.0: # %entry
-; MMR3-NEXT: addiusp -40
-; MMR3-NEXT: .cfi_def_cfa_offset 40
+; MMR3-NEXT: addiusp -48
+; MMR3-NEXT: .cfi_def_cfa_offset 48
+; MMR3-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; MMR3-NEXT: swp $16, 32($sp)
-; MMR3-NEXT: .cfi_offset 17, -4
-; MMR3-NEXT: .cfi_offset 16, -8
+; MMR3-NEXT: .cfi_offset 31, -4
+; MMR3-NEXT: .cfi_offset 30, -8
+; MMR3-NEXT: .cfi_offset 17, -12
+; MMR3-NEXT: .cfi_offset 16, -16
+; MMR3-NEXT: move $fp, $sp
+; MMR3-NEXT: .cfi_def_cfa_register 30
+; MMR3-NEXT: addiu $1, $zero, -16
+; MMR3-NEXT: and $sp, $sp, $1
; MMR3-NEXT: li16 $2, 0
; MMR3-NEXT: swp $6, 24($sp)
; MMR3-NEXT: swp $4, 16($sp)
@@ -648,7 +700,7 @@ define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
; MMR3-NEXT: sw $2, 0($sp)
; MMR3-NEXT: addiur1sp $2, 0
; MMR3-NEXT: addiur2 $2, $2, 16
-; MMR3-NEXT: lw $3, 68($sp)
+; MMR3-NEXT: lw $3, 76($fp)
; MMR3-NEXT: srl16 $4, $3, 3
; MMR3-NEXT: andi $4, $4, 12
; MMR3-NEXT: subu16 $5, $2, $4
@@ -670,16 +722,27 @@ define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
; MMR3-NEXT: sllv $5, $5, $1
; MMR3-NEXT: or16 $5, $6
; MMR3-NEXT: srlv $2, $17, $2
+; MMR3-NEXT: move $sp, $fp
; MMR3-NEXT: lwp $16, 32($sp)
-; MMR3-NEXT: addiusp 40
+; MMR3-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR3-NEXT: addiusp 48
; MMR3-NEXT: jrc $ra
;
; MMR6-LABEL: lshr_i128:
; MMR6: # %bb.0: # %entry
-; MMR6-NEXT: addiu $sp, $sp, -40
-; MMR6-NEXT: .cfi_def_cfa_offset 40
+; MMR6-NEXT: addiu $sp, $sp, -48
+; MMR6-NEXT: .cfi_def_cfa_offset 48
+; MMR6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; MMR6-NEXT: sw $16, 36($sp) # 4-byte Folded Spill
-; MMR6-NEXT: .cfi_offset 16, -4
+; MMR6-NEXT: .cfi_offset 31, -4
+; MMR6-NEXT: .cfi_offset 30, -8
+; MMR6-NEXT: .cfi_offset 16, -12
+; MMR6-NEXT: move $fp, $sp
+; MMR6-NEXT: .cfi_def_cfa_register 30
+; MMR6-NEXT: addiu $1, $zero, -16
+; MMR6-NEXT: and $sp, $sp, $1
; MMR6-NEXT: li16 $2, 0
; MMR6-NEXT: sw $7, 28($sp)
; MMR6-NEXT: sw $6, 24($sp)
@@ -691,7 +754,7 @@ define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
; MMR6-NEXT: sw $2, 0($sp)
; MMR6-NEXT: addiu $2, $sp, 0
; MMR6-NEXT: addiur2 $2, $2, 16
-; MMR6-NEXT: lw $3, 68($sp)
+; MMR6-NEXT: lw $3, 76($fp)
; MMR6-NEXT: srl16 $4, $3, 3
; MMR6-NEXT: andi $4, $4, 12
; MMR6-NEXT: subu16 $2, $2, $4
@@ -714,8 +777,11 @@ define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
; MMR6-NEXT: sllv $2, $2, $7
; MMR6-NEXT: or $5, $1, $2
; MMR6-NEXT: srlv $2, $16, $6
+; MMR6-NEXT: move $sp, $fp
; MMR6-NEXT: lw $16, 36($sp) # 4-byte Folded Reload
-; MMR6-NEXT: addiu $sp, $sp, 40
+; MMR6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR6-NEXT: addiu $sp, $sp, 48
; MMR6-NEXT: jrc $ra
entry:
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/not.ll b/llvm/test/CodeGen/Mips/llvm-ir/not.ll
index 4fa50742e9acd..c1604b797cb0b 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/not.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/not.ll
@@ -1,3 +1,4 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 6
; RUN: llc < %s -mtriple=mips-unknown-linux-gnu -mcpu=mips2 | FileCheck %s -check-prefixes=GP32
; RUN: llc < %s -mtriple=mips-unknown-linux-gnu -mcpu=mips32 | FileCheck %s -check-prefixes=GP32
; RUN: llc < %s -mtriple=mips-unknown-linux-gnu -mcpu=mips32r2 | FileCheck %s \
@@ -301,15 +302,29 @@ entry:
define signext i128 @nor_i128(i128 signext %a, i128 signext %b) {
; GP32-LABEL: nor_i128:
; GP32: # %bb.0: # %entry
-; GP32-NEXT: lw $1, 20($sp)
-; GP32-NEXT: lw $2, 16($sp)
+; GP32-NEXT: addiu $sp, $sp, -16
+; GP32-NEXT: .cfi_def_cfa_offset 16
+; GP32-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; GP32-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; GP32-NEXT: .cfi_offset 31, -4
+; GP32-NEXT: .cfi_offset 30, -8
+; GP32-NEXT: move $fp, $sp
+; GP32-NEXT: .cfi_def_cfa_register 30
+; GP32-NEXT: addiu $1, $zero, -16
+; GP32-NEXT: and $sp, $sp, $1
+; GP32-NEXT: lw $1, 36($fp)
+; GP32-NEXT: lw $2, 32($fp)
; GP32-NEXT: nor $2, $2, $4
; GP32-NEXT: nor $3, $1, $5
-; GP32-NEXT: lw $1, 24($sp)
+; GP32-NEXT: lw $1, 40($fp)
; GP32-NEXT: nor $4, $1, $6
-; GP32-NEXT: lw $1, 28($sp)
-; GP32-NEXT: jr $ra
+; GP32-NEXT: lw $1, 44($fp)
; GP32-NEXT: nor $5, $1, $7
+; GP32-NEXT: move $sp, $fp
+; GP32-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; GP32-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; GP32-NEXT: jr $ra
+; GP32-NEXT: addiu $sp, $sp, 16
;
; GP64-LABEL: nor_i128:
; GP64: # %bb.0: # %entry
@@ -319,26 +334,54 @@ define signext i128 @nor_i128(i128 signext %a, i128 signext %b) {
;
; MM32r3-LABEL: nor_i128:
; MM32r3: # %bb.0: # %entry
-; MM32r3-NEXT: lw $1, 20($sp)
-; MM32r3-NEXT: lw $2, 16($sp)
+; MM32r3-NEXT: addiusp -16
+; MM32r3-NEXT: .cfi_def_cfa_offset 16
+; MM32r3-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MM32r3-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MM32r3-NEXT: .cfi_offset 31, -4
+; MM32r3-NEXT: .cfi_offset 30, -8
+; MM32r3-NEXT: move $fp, $sp
+; MM32r3-NEXT: .cfi_def_cfa_register 30
+; MM32r3-NEXT: addiu $1, $zero, -16
+; MM32r3-NEXT: and $sp, $sp, $1
+; MM32r3-NEXT: lw $1, 36($fp)
+; MM32r3-NEXT: lw $2, 32($fp)
; MM32r3-NEXT: nor $2, $2, $4
; MM32r3-NEXT: nor $3, $1, $5
-; MM32r3-NEXT: lw $1, 24($sp)
+; MM32r3-NEXT: lw $1, 40($fp)
; MM32r3-NEXT: nor $4, $1, $6
-; MM32r3-NEXT: lw $1, 28($sp)
-; MM32r3-NEXT: jr $ra
+; MM32r3-NEXT: lw $1, 44($fp)
; MM32r3-NEXT: nor $5, $1, $7
+; MM32r3-NEXT: move $sp, $fp
+; MM32r3-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MM32r3-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MM32r3-NEXT: addiusp 16
+; MM32r3-NEXT: jrc $ra
;
; MM32r6-LABEL: nor_i128:
; MM32r6: # %bb.0: # %entry
-; MM32r6-NEXT: lw $1, 20($sp)
-; MM32r6-NEXT: lw $2, 16($sp)
+; MM32r6-NEXT: addiu $sp, $sp, -16
+; MM32r6-NEXT: .cfi_def_cfa_offset 16
+; MM32r6-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MM32r6-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MM32r6-NEXT: .cfi_offset 31, -4
+; MM32r6-NEXT: .cfi_offset 30, -8
+; MM32r6-NEXT: move $fp, $sp
+; MM32r6-NEXT: .cfi_def_cfa_register 30
+; MM32r6-NEXT: addiu $1, $zero, -16
+; MM32r6-NEXT: and $sp, $sp, $1
+; MM32r6-NEXT: lw $1, 36($fp)
+; MM32r6-NEXT: lw $2, 32($fp)
; MM32r6-NEXT: nor $2, $2, $4
; MM32r6-NEXT: nor $3, $1, $5
-; MM32r6-NEXT: lw $1, 24($sp)
+; MM32r6-NEXT: lw $1, 40($fp)
; MM32r6-NEXT: nor $4, $1, $6
-; MM32r6-NEXT: lw $1, 28($sp)
+; MM32r6-NEXT: lw $1, 44($fp)
; MM32r6-NEXT: nor $5, $1, $7
+; MM32r6-NEXT: move $sp, $fp
+; MM32r6-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MM32r6-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MM32r6-NEXT: addiu $sp, $sp, 16
; MM32r6-NEXT: jrc $ra
entry:
%or = or i128 %b, %a
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/or.ll b/llvm/test/CodeGen/Mips/llvm-ir/or.ll
index c595ff42086b2..061c4ff800bb9 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/or.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/or.ll
@@ -157,15 +157,29 @@ entry:
define signext i128 @or_i128(i128 signext %a, i128 signext %b) {
; GP32-LABEL: or_i128:
; GP32: # %bb.0: # %entry
-; GP32-NEXT: lw $1, 20($sp)
-; GP32-NEXT: lw $2, 16($sp)
+; GP32-NEXT: addiu $sp, $sp, -16
+; GP32-NEXT: .cfi_def_cfa_offset 16
+; GP32-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; GP32-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; GP32-NEXT: .cfi_offset 31, -4
+; GP32-NEXT: .cfi_offset 30, -8
+; GP32-NEXT: move $fp, $sp
+; GP32-NEXT: .cfi_def_cfa_register 30
+; GP32-NEXT: addiu $1, $zero, -16
+; GP32-NEXT: and $sp, $sp, $1
+; GP32-NEXT: lw $1, 36($fp)
+; GP32-NEXT: lw $2, 32($fp)
; GP32-NEXT: or $2, $4, $2
; GP32-NEXT: or $3, $5, $1
-; GP32-NEXT: lw $1, 24($sp)
+; GP32-NEXT: lw $1, 40($fp)
; GP32-NEXT: or $4, $6, $1
-; GP32-NEXT: lw $1, 28($sp)
-; GP32-NEXT: jr $ra
+; GP32-NEXT: lw $1, 44($fp)
; GP32-NEXT: or $5, $7, $1
+; GP32-NEXT: move $sp, $fp
+; GP32-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; GP32-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; GP32-NEXT: jr $ra
+; GP32-NEXT: addiu $sp, $sp, 16
;
; GP64-LABEL: or_i128:
; GP64: # %bb.0: # %entry
@@ -175,25 +189,53 @@ define signext i128 @or_i128(i128 signext %a, i128 signext %b) {
;
; MM32-LABEL: or_i128:
; MM32: # %bb.0: # %entry
-; MM32-NEXT: lwp $2, 16($sp)
+; MM32-NEXT: addiusp -16
+; MM32-NEXT: .cfi_def_cfa_offset 16
+; MM32-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MM32-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MM32-NEXT: .cfi_offset 31, -4
+; MM32-NEXT: .cfi_offset 30, -8
+; MM32-NEXT: move $fp, $sp
+; MM32-NEXT: .cfi_def_cfa_register 30
+; MM32-NEXT: addiu $1, $zero, -16
+; MM32-NEXT: and $sp, $sp, $1
+; MM32-NEXT: lwp $2, 32($fp)
; MM32-NEXT: or16 $2, $4
; MM32-NEXT: or16 $3, $5
-; MM32-NEXT: lw $4, 24($sp)
+; MM32-NEXT: lw $4, 40($fp)
; MM32-NEXT: or16 $4, $6
-; MM32-NEXT: lw $5, 28($sp)
+; MM32-NEXT: lw $5, 44($fp)
; MM32-NEXT: or16 $5, $7
+; MM32-NEXT: move $sp, $fp
+; MM32-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MM32-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MM32-NEXT: addiusp 16
; MM32-NEXT: jrc $ra
;
; MM32R6-LABEL: or_i128:
; MM32R6: # %bb.0: # %entry
-; MM32R6-NEXT: lw $1, 20($sp)
-; MM32R6-NEXT: lw $2, 16($sp)
+; MM32R6-NEXT: addiu $sp, $sp, -16
+; MM32R6-NEXT: .cfi_def_cfa_offset 16
+; MM32R6-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MM32R6-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MM32R6-NEXT: .cfi_offset 31, -4
+; MM32R6-NEXT: .cfi_offset 30, -8
+; MM32R6-NEXT: move $fp, $sp
+; MM32R6-NEXT: .cfi_def_cfa_register 30
+; MM32R6-NEXT: addiu $1, $zero, -16
+; MM32R6-NEXT: and $sp, $sp, $1
+; MM32R6-NEXT: lw $1, 36($fp)
+; MM32R6-NEXT: lw $2, 32($fp)
; MM32R6-NEXT: or $2, $4, $2
; MM32R6-NEXT: or $3, $5, $1
-; MM32R6-NEXT: lw $1, 24($sp)
+; MM32R6-NEXT: lw $1, 40($fp)
; MM32R6-NEXT: or $4, $6, $1
-; MM32R6-NEXT: lw $1, 28($sp)
+; MM32R6-NEXT: lw $1, 44($fp)
; MM32R6-NEXT: or $5, $7, $1
+; MM32R6-NEXT: move $sp, $fp
+; MM32R6-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MM32R6-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MM32R6-NEXT: addiu $sp, $sp, 16
; MM32R6-NEXT: jrc $ra
entry:
%r = or i128 %a, %b
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/sdiv.ll b/llvm/test/CodeGen/Mips/llvm-ir/sdiv.ll
index 72cead18f89fa..48a9ea17eff3e 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/sdiv.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/sdiv.ll
@@ -420,72 +420,96 @@ define signext i128 @sdiv_i128(i128 signext %a, i128 signext %b) {
; MIPS2: # %bb.0: # %entry
; MIPS2-NEXT: lui $2, %hi(_gp_disp)
; MIPS2-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MIPS2-NEXT: addiu $sp, $sp, -40
-; MIPS2-NEXT: .cfi_def_cfa_offset 40
-; MIPS2-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: addiu $sp, $sp, -48
+; MIPS2-NEXT: .cfi_def_cfa_offset 48
+; MIPS2-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; MIPS2-NEXT: .cfi_offset 31, -4
+; MIPS2-NEXT: .cfi_offset 30, -8
+; MIPS2-NEXT: move $fp, $sp
+; MIPS2-NEXT: .cfi_def_cfa_register 30
+; MIPS2-NEXT: addiu $1, $zero, -16
+; MIPS2-NEXT: and $sp, $sp, $1
; MIPS2-NEXT: addu $gp, $2, $25
-; MIPS2-NEXT: lw $1, 60($sp)
-; MIPS2-NEXT: lw $2, 64($sp)
-; MIPS2-NEXT: lw $3, 68($sp)
+; MIPS2-NEXT: lw $1, 68($fp)
+; MIPS2-NEXT: lw $2, 72($fp)
+; MIPS2-NEXT: lw $3, 76($fp)
; MIPS2-NEXT: sw $3, 28($sp)
; MIPS2-NEXT: sw $2, 24($sp)
; MIPS2-NEXT: sw $1, 20($sp)
-; MIPS2-NEXT: lw $1, 56($sp)
+; MIPS2-NEXT: lw $1, 64($fp)
; MIPS2-NEXT: sw $1, 16($sp)
; MIPS2-NEXT: lw $25, %call16(__divti3)($gp)
; MIPS2-NEXT: jalr $25
; MIPS2-NEXT: nop
-; MIPS2-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: move $sp, $fp
+; MIPS2-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS2-NEXT: jr $ra
-; MIPS2-NEXT: addiu $sp, $sp, 40
+; MIPS2-NEXT: addiu $sp, $sp, 48
;
; GP32-LABEL: sdiv_i128:
; GP32: # %bb.0: # %entry
; GP32-NEXT: lui $2, %hi(_gp_disp)
; GP32-NEXT: addiu $2, $2, %lo(_gp_disp)
-; GP32-NEXT: addiu $sp, $sp, -40
-; GP32-NEXT: .cfi_def_cfa_offset 40
-; GP32-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; GP32-NEXT: addiu $sp, $sp, -48
+; GP32-NEXT: .cfi_def_cfa_offset 48
+; GP32-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; GP32-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; GP32-NEXT: .cfi_offset 31, -4
+; GP32-NEXT: .cfi_offset 30, -8
+; GP32-NEXT: move $fp, $sp
+; GP32-NEXT: .cfi_def_cfa_register 30
+; GP32-NEXT: addiu $1, $zero, -16
+; GP32-NEXT: and $sp, $sp, $1
; GP32-NEXT: addu $gp, $2, $25
-; GP32-NEXT: lw $1, 60($sp)
-; GP32-NEXT: lw $2, 64($sp)
-; GP32-NEXT: lw $3, 68($sp)
+; GP32-NEXT: lw $1, 68($fp)
+; GP32-NEXT: lw $2, 72($fp)
+; GP32-NEXT: lw $3, 76($fp)
; GP32-NEXT: sw $3, 28($sp)
; GP32-NEXT: sw $2, 24($sp)
; GP32-NEXT: sw $1, 20($sp)
-; GP32-NEXT: lw $1, 56($sp)
+; GP32-NEXT: lw $1, 64($fp)
; GP32-NEXT: sw $1, 16($sp)
; GP32-NEXT: lw $25, %call16(__divti3)($gp)
; GP32-NEXT: jalr $25
; GP32-NEXT: nop
-; GP32-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; GP32-NEXT: move $sp, $fp
+; GP32-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; GP32-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; GP32-NEXT: jr $ra
-; GP32-NEXT: addiu $sp, $sp, 40
+; GP32-NEXT: addiu $sp, $sp, 48
;
; GP32R6-LABEL: sdiv_i128:
; GP32R6: # %bb.0: # %entry
; GP32R6-NEXT: lui $2, %hi(_gp_disp)
; GP32R6-NEXT: addiu $2, $2, %lo(_gp_disp)
-; GP32R6-NEXT: addiu $sp, $sp, -40
-; GP32R6-NEXT: .cfi_def_cfa_offset 40
-; GP32R6-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; GP32R6-NEXT: addiu $sp, $sp, -48
+; GP32R6-NEXT: .cfi_def_cfa_offset 48
+; GP32R6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; GP32R6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; GP32R6-NEXT: .cfi_offset 31, -4
+; GP32R6-NEXT: .cfi_offset 30, -8
+; GP32R6-NEXT: move $fp, $sp
+; GP32R6-NEXT: .cfi_def_cfa_register 30
+; GP32R6-NEXT: addiu $1, $zero, -16
+; GP32R6-NEXT: and $sp, $sp, $1
; GP32R6-NEXT: addu $gp, $2, $25
-; GP32R6-NEXT: lw $1, 60($sp)
-; GP32R6-NEXT: lw $2, 64($sp)
-; GP32R6-NEXT: lw $3, 68($sp)
+; GP32R6-NEXT: lw $1, 68($fp)
+; GP32R6-NEXT: lw $2, 72($fp)
+; GP32R6-NEXT: lw $3, 76($fp)
; GP32R6-NEXT: sw $3, 28($sp)
; GP32R6-NEXT: sw $2, 24($sp)
; GP32R6-NEXT: sw $1, 20($sp)
-; GP32R6-NEXT: lw $1, 56($sp)
+; GP32R6-NEXT: lw $1, 64($fp)
; GP32R6-NEXT: sw $1, 16($sp)
; GP32R6-NEXT: lw $25, %call16(__divti3)($gp)
; GP32R6-NEXT: jalrc $25
-; GP32R6-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; GP32R6-NEXT: move $sp, $fp
+; GP32R6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; GP32R6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; GP32R6-NEXT: jr $ra
-; GP32R6-NEXT: addiu $sp, $sp, 40
+; GP32R6-NEXT: addiu $sp, $sp, 48
;
; MIPS3-LABEL: sdiv_i128:
; MIPS3: # %bb.0: # %entry
@@ -547,59 +571,75 @@ define signext i128 @sdiv_i128(i128 signext %a, i128 signext %b) {
; MMR3: # %bb.0: # %entry
; MMR3-NEXT: lui $2, %hi(_gp_disp)
; MMR3-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MMR3-NEXT: addiusp -40
-; MMR3-NEXT: .cfi_def_cfa_offset 40
-; MMR3-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
-; MMR3-NEXT: sw $17, 32($sp) # 4-byte Folded Spill
+; MMR3-NEXT: addiusp -48
+; MMR3-NEXT: .cfi_def_cfa_offset 48
+; MMR3-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $17, 36($sp) # 4-byte Folded Spill
; MMR3-NEXT: .cfi_offset 31, -4
-; MMR3-NEXT: .cfi_offset 17, -8
+; MMR3-NEXT: .cfi_offset 30, -8
+; MMR3-NEXT: .cfi_offset 17, -12
+; MMR3-NEXT: move $fp, $sp
+; MMR3-NEXT: .cfi_def_cfa_register 30
+; MMR3-NEXT: addiu $1, $zero, -16
+; MMR3-NEXT: and $sp, $sp, $1
; MMR3-NEXT: addu $gp, $2, $25
; MMR3-NEXT: move $1, $7
-; MMR3-NEXT: lw $7, 60($sp)
-; MMR3-NEXT: lw $17, 64($sp)
-; MMR3-NEXT: lw $3, 68($sp)
+; MMR3-NEXT: lw $7, 68($fp)
+; MMR3-NEXT: lw $17, 72($fp)
+; MMR3-NEXT: lw $3, 76($fp)
; MMR3-NEXT: move $2, $sp
; MMR3-NEXT: sw16 $3, 28($2)
; MMR3-NEXT: sw16 $17, 24($2)
; MMR3-NEXT: sw16 $7, 20($2)
-; MMR3-NEXT: lw $3, 56($sp)
+; MMR3-NEXT: lw $3, 64($fp)
; MMR3-NEXT: sw16 $3, 16($2)
; MMR3-NEXT: lw $25, %call16(__divti3)($gp)
; MMR3-NEXT: move $7, $1
; MMR3-NEXT: jalr $25
; MMR3-NEXT: nop
-; MMR3-NEXT: lw $17, 32($sp) # 4-byte Folded Reload
-; MMR3-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
-; MMR3-NEXT: addiusp 40
+; MMR3-NEXT: move $sp, $fp
+; MMR3-NEXT: lw $17, 36($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR3-NEXT: addiusp 48
; MMR3-NEXT: jrc $ra
;
; MMR6-LABEL: sdiv_i128:
; MMR6: # %bb.0: # %entry
; MMR6-NEXT: lui $2, %hi(_gp_disp)
; MMR6-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MMR6-NEXT: addiu $sp, $sp, -40
-; MMR6-NEXT: .cfi_def_cfa_offset 40
-; MMR6-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
-; MMR6-NEXT: sw $17, 32($sp) # 4-byte Folded Spill
+; MMR6-NEXT: addiu $sp, $sp, -48
+; MMR6-NEXT: .cfi_def_cfa_offset 48
+; MMR6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $17, 36($sp) # 4-byte Folded Spill
; MMR6-NEXT: .cfi_offset 31, -4
-; MMR6-NEXT: .cfi_offset 17, -8
+; MMR6-NEXT: .cfi_offset 30, -8
+; MMR6-NEXT: .cfi_offset 17, -12
+; MMR6-NEXT: move $fp, $sp
+; MMR6-NEXT: .cfi_def_cfa_register 30
+; MMR6-NEXT: addiu $1, $zero, -16
+; MMR6-NEXT: and $sp, $sp, $1
; MMR6-NEXT: addu $gp, $2, $25
; MMR6-NEXT: move $1, $7
-; MMR6-NEXT: lw $7, 60($sp)
-; MMR6-NEXT: lw $17, 64($sp)
-; MMR6-NEXT: lw $3, 68($sp)
+; MMR6-NEXT: lw $7, 68($fp)
+; MMR6-NEXT: lw $17, 72($fp)
+; MMR6-NEXT: lw $3, 76($fp)
; MMR6-NEXT: move $2, $sp
; MMR6-NEXT: sw16 $3, 28($2)
; MMR6-NEXT: sw16 $17, 24($2)
; MMR6-NEXT: sw16 $7, 20($2)
-; MMR6-NEXT: lw $3, 56($sp)
+; MMR6-NEXT: lw $3, 64($fp)
; MMR6-NEXT: sw16 $3, 16($2)
; MMR6-NEXT: lw $25, %call16(__divti3)($gp)
; MMR6-NEXT: move $7, $1
; MMR6-NEXT: jalr $25
-; MMR6-NEXT: lw $17, 32($sp) # 4-byte Folded Reload
-; MMR6-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
-; MMR6-NEXT: addiu $sp, $sp, 40
+; MMR6-NEXT: move $sp, $fp
+; MMR6-NEXT: lw $17, 36($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR6-NEXT: addiu $sp, $sp, 48
; MMR6-NEXT: jrc $ra
entry:
%r = sdiv i128 %a, %b
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/shl.ll b/llvm/test/CodeGen/Mips/llvm-ir/shl.ll
index 394890a9dcc7c..d8e55d465a6e2 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/shl.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/shl.ll
@@ -438,9 +438,17 @@ entry:
define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MIPS2-LABEL: shl_i128:
; MIPS2: # %bb.0: # %entry
-; MIPS2-NEXT: addiu $sp, $sp, -32
-; MIPS2-NEXT: .cfi_def_cfa_offset 32
-; MIPS2-NEXT: lw $1, 60($sp)
+; MIPS2-NEXT: addiu $sp, $sp, -48
+; MIPS2-NEXT: .cfi_def_cfa_offset 48
+; MIPS2-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: .cfi_offset 31, -4
+; MIPS2-NEXT: .cfi_offset 30, -8
+; MIPS2-NEXT: move $fp, $sp
+; MIPS2-NEXT: .cfi_def_cfa_register 30
+; MIPS2-NEXT: addiu $1, $zero, -16
+; MIPS2-NEXT: and $sp, $sp, $1
+; MIPS2-NEXT: lw $1, 76($fp)
; MIPS2-NEXT: srl $2, $1, 3
; MIPS2-NEXT: sw $7, 12($sp)
; MIPS2-NEXT: sw $6, 8($sp)
@@ -448,7 +456,7 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MIPS2-NEXT: sw $4, 0($sp)
; MIPS2-NEXT: andi $2, $2, 12
; MIPS2-NEXT: addiu $3, $sp, 0
-; MIPS2-NEXT: addu $4, $3, $2
+; MIPS2-NEXT: or $4, $3, $2
; MIPS2-NEXT: sw $zero, 28($sp)
; MIPS2-NEXT: sw $zero, 24($sp)
; MIPS2-NEXT: sw $zero, 20($sp)
@@ -472,14 +480,25 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MIPS2-NEXT: srlv $4, $4, $7
; MIPS2-NEXT: or $4, $5, $4
; MIPS2-NEXT: sllv $5, $6, $1
+; MIPS2-NEXT: move $sp, $fp
+; MIPS2-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS2-NEXT: jr $ra
-; MIPS2-NEXT: addiu $sp, $sp, 32
+; MIPS2-NEXT: addiu $sp, $sp, 48
;
; MIPS32-LABEL: shl_i128:
; MIPS32: # %bb.0: # %entry
-; MIPS32-NEXT: addiu $sp, $sp, -32
-; MIPS32-NEXT: .cfi_def_cfa_offset 32
-; MIPS32-NEXT: lw $1, 60($sp)
+; MIPS32-NEXT: addiu $sp, $sp, -48
+; MIPS32-NEXT: .cfi_def_cfa_offset 48
+; MIPS32-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS32-NEXT: .cfi_offset 31, -4
+; MIPS32-NEXT: .cfi_offset 30, -8
+; MIPS32-NEXT: move $fp, $sp
+; MIPS32-NEXT: .cfi_def_cfa_register 30
+; MIPS32-NEXT: addiu $1, $zero, -16
+; MIPS32-NEXT: and $sp, $sp, $1
+; MIPS32-NEXT: lw $1, 76($fp)
; MIPS32-NEXT: srl $2, $1, 3
; MIPS32-NEXT: sw $7, 12($sp)
; MIPS32-NEXT: sw $6, 8($sp)
@@ -487,7 +506,7 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MIPS32-NEXT: sw $4, 0($sp)
; MIPS32-NEXT: andi $2, $2, 12
; MIPS32-NEXT: addiu $3, $sp, 0
-; MIPS32-NEXT: addu $4, $3, $2
+; MIPS32-NEXT: or $4, $3, $2
; MIPS32-NEXT: sw $zero, 28($sp)
; MIPS32-NEXT: sw $zero, 24($sp)
; MIPS32-NEXT: sw $zero, 20($sp)
@@ -511,14 +530,25 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MIPS32-NEXT: srlv $4, $4, $7
; MIPS32-NEXT: or $4, $5, $4
; MIPS32-NEXT: sllv $5, $6, $1
+; MIPS32-NEXT: move $sp, $fp
+; MIPS32-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS32-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS32-NEXT: jr $ra
-; MIPS32-NEXT: addiu $sp, $sp, 32
+; MIPS32-NEXT: addiu $sp, $sp, 48
;
; MIPS32R2-LABEL: shl_i128:
; MIPS32R2: # %bb.0: # %entry
-; MIPS32R2-NEXT: addiu $sp, $sp, -32
-; MIPS32R2-NEXT: .cfi_def_cfa_offset 32
-; MIPS32R2-NEXT: lw $1, 60($sp)
+; MIPS32R2-NEXT: addiu $sp, $sp, -48
+; MIPS32R2-NEXT: .cfi_def_cfa_offset 48
+; MIPS32R2-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS32R2-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS32R2-NEXT: .cfi_offset 31, -4
+; MIPS32R2-NEXT: .cfi_offset 30, -8
+; MIPS32R2-NEXT: move $fp, $sp
+; MIPS32R2-NEXT: .cfi_def_cfa_register 30
+; MIPS32R2-NEXT: addiu $1, $zero, -16
+; MIPS32R2-NEXT: and $sp, $sp, $1
+; MIPS32R2-NEXT: lw $1, 76($fp)
; MIPS32R2-NEXT: srl $2, $1, 3
; MIPS32R2-NEXT: sw $7, 12($sp)
; MIPS32R2-NEXT: sw $6, 8($sp)
@@ -526,7 +556,7 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MIPS32R2-NEXT: sw $4, 0($sp)
; MIPS32R2-NEXT: andi $2, $2, 12
; MIPS32R2-NEXT: addiu $3, $sp, 0
-; MIPS32R2-NEXT: addu $4, $3, $2
+; MIPS32R2-NEXT: or $4, $3, $2
; MIPS32R2-NEXT: sw $zero, 28($sp)
; MIPS32R2-NEXT: sw $zero, 24($sp)
; MIPS32R2-NEXT: sw $zero, 20($sp)
@@ -550,14 +580,25 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MIPS32R2-NEXT: srlv $4, $4, $7
; MIPS32R2-NEXT: or $4, $5, $4
; MIPS32R2-NEXT: sllv $5, $6, $1
+; MIPS32R2-NEXT: move $sp, $fp
+; MIPS32R2-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS32R2-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS32R2-NEXT: jr $ra
-; MIPS32R2-NEXT: addiu $sp, $sp, 32
+; MIPS32R2-NEXT: addiu $sp, $sp, 48
;
; MIPS32R6-LABEL: shl_i128:
; MIPS32R6: # %bb.0: # %entry
-; MIPS32R6-NEXT: addiu $sp, $sp, -32
-; MIPS32R6-NEXT: .cfi_def_cfa_offset 32
-; MIPS32R6-NEXT: lw $1, 60($sp)
+; MIPS32R6-NEXT: addiu $sp, $sp, -48
+; MIPS32R6-NEXT: .cfi_def_cfa_offset 48
+; MIPS32R6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS32R6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MIPS32R6-NEXT: .cfi_offset 31, -4
+; MIPS32R6-NEXT: .cfi_offset 30, -8
+; MIPS32R6-NEXT: move $fp, $sp
+; MIPS32R6-NEXT: .cfi_def_cfa_register 30
+; MIPS32R6-NEXT: addiu $1, $zero, -16
+; MIPS32R6-NEXT: and $sp, $sp, $1
+; MIPS32R6-NEXT: lw $1, 76($fp)
; MIPS32R6-NEXT: srl $2, $1, 3
; MIPS32R6-NEXT: sw $7, 12($sp)
; MIPS32R6-NEXT: sw $6, 8($sp)
@@ -565,7 +606,7 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MIPS32R6-NEXT: sw $4, 0($sp)
; MIPS32R6-NEXT: andi $2, $2, 12
; MIPS32R6-NEXT: addiu $3, $sp, 0
-; MIPS32R6-NEXT: addu $4, $3, $2
+; MIPS32R6-NEXT: or $4, $3, $2
; MIPS32R6-NEXT: sw $zero, 28($sp)
; MIPS32R6-NEXT: sw $zero, 24($sp)
; MIPS32R6-NEXT: sw $zero, 20($sp)
@@ -589,8 +630,11 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MIPS32R6-NEXT: srlv $4, $4, $7
; MIPS32R6-NEXT: or $4, $5, $4
; MIPS32R6-NEXT: sllv $5, $6, $1
+; MIPS32R6-NEXT: move $sp, $fp
+; MIPS32R6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS32R6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS32R6-NEXT: jr $ra
-; MIPS32R6-NEXT: addiu $sp, $sp, 32
+; MIPS32R6-NEXT: addiu $sp, $sp, 48
;
; MIPS3-LABEL: shl_i128:
; MIPS3: # %bb.0: # %entry
@@ -678,11 +722,19 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
;
; MMR3-LABEL: shl_i128:
; MMR3: # %bb.0: # %entry
-; MMR3-NEXT: addiusp -40
-; MMR3-NEXT: .cfi_def_cfa_offset 40
+; MMR3-NEXT: addiusp -48
+; MMR3-NEXT: .cfi_def_cfa_offset 48
+; MMR3-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; MMR3-NEXT: swp $16, 32($sp)
-; MMR3-NEXT: .cfi_offset 17, -4
-; MMR3-NEXT: .cfi_offset 16, -8
+; MMR3-NEXT: .cfi_offset 31, -4
+; MMR3-NEXT: .cfi_offset 30, -8
+; MMR3-NEXT: .cfi_offset 17, -12
+; MMR3-NEXT: .cfi_offset 16, -16
+; MMR3-NEXT: move $fp, $sp
+; MMR3-NEXT: .cfi_def_cfa_register 30
+; MMR3-NEXT: addiu $1, $zero, -16
+; MMR3-NEXT: and $sp, $sp, $1
; MMR3-NEXT: li16 $2, 0
; MMR3-NEXT: sw $2, 28($sp)
; MMR3-NEXT: sw $2, 24($sp)
@@ -690,11 +742,11 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MMR3-NEXT: sw $2, 16($sp)
; MMR3-NEXT: swp $6, 8($sp)
; MMR3-NEXT: swp $4, 0($sp)
-; MMR3-NEXT: lw $2, 68($sp)
+; MMR3-NEXT: lw $2, 76($fp)
; MMR3-NEXT: srl16 $3, $2, 3
; MMR3-NEXT: andi $3, $3, 12
; MMR3-NEXT: addiur1sp $4, 0
-; MMR3-NEXT: addu16 $4, $4, $3
+; MMR3-NEXT: or16 $4, $3
; MMR3-NEXT: lw16 $6, 8($4)
; MMR3-NEXT: lw16 $7, 4($4)
; MMR3-NEXT: andi16 $5, $2, 31
@@ -714,14 +766,25 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MMR3-NEXT: srlv $4, $4, $1
; MMR3-NEXT: or16 $4, $6
; MMR3-NEXT: sllv $5, $7, $5
+; MMR3-NEXT: move $sp, $fp
; MMR3-NEXT: lwp $16, 32($sp)
-; MMR3-NEXT: addiusp 40
+; MMR3-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR3-NEXT: addiusp 48
; MMR3-NEXT: jrc $ra
;
; MMR6-LABEL: shl_i128:
; MMR6: # %bb.0: # %entry
-; MMR6-NEXT: addiu $sp, $sp, -32
-; MMR6-NEXT: .cfi_def_cfa_offset 32
+; MMR6-NEXT: addiu $sp, $sp, -48
+; MMR6-NEXT: .cfi_def_cfa_offset 48
+; MMR6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MMR6-NEXT: .cfi_offset 31, -4
+; MMR6-NEXT: .cfi_offset 30, -8
+; MMR6-NEXT: move $fp, $sp
+; MMR6-NEXT: .cfi_def_cfa_register 30
+; MMR6-NEXT: addiu $1, $zero, -16
+; MMR6-NEXT: and $sp, $sp, $1
; MMR6-NEXT: li16 $2, 0
; MMR6-NEXT: sw $2, 28($sp)
; MMR6-NEXT: sw $2, 24($sp)
@@ -731,11 +794,11 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MMR6-NEXT: sw $6, 8($sp)
; MMR6-NEXT: sw $5, 4($sp)
; MMR6-NEXT: sw $4, 0($sp)
-; MMR6-NEXT: lw $2, 60($sp)
+; MMR6-NEXT: lw $2, 76($fp)
; MMR6-NEXT: srl16 $3, $2, 3
-; MMR6-NEXT: andi $3, $3, 12
-; MMR6-NEXT: addiu $4, $sp, 0
-; MMR6-NEXT: addu16 $4, $4, $3
+; MMR6-NEXT: andi $1, $3, 12
+; MMR6-NEXT: addiu $3, $sp, 0
+; MMR6-NEXT: or $4, $3, $1
; MMR6-NEXT: lw16 $5, 8($4)
; MMR6-NEXT: lw16 $3, 4($4)
; MMR6-NEXT: andi16 $6, $2, 31
@@ -755,7 +818,10 @@ define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
; MMR6-NEXT: srlv $4, $4, $7
; MMR6-NEXT: or $4, $1, $4
; MMR6-NEXT: sllv $5, $5, $6
-; MMR6-NEXT: addiu $sp, $sp, 32
+; MMR6-NEXT: move $sp, $fp
+; MMR6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR6-NEXT: addiu $sp, $sp, 48
; MMR6-NEXT: jrc $ra
entry:
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/srem.ll b/llvm/test/CodeGen/Mips/llvm-ir/srem.ll
index 72496fcc53a5a..1176b0bb4af9d 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/srem.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/srem.ll
@@ -368,72 +368,96 @@ define signext i128 @srem_i128(i128 signext %a, i128 signext %b) {
; MIPS2: # %bb.0: # %entry
; MIPS2-NEXT: lui $2, %hi(_gp_disp)
; MIPS2-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MIPS2-NEXT: addiu $sp, $sp, -40
-; MIPS2-NEXT: .cfi_def_cfa_offset 40
-; MIPS2-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: addiu $sp, $sp, -48
+; MIPS2-NEXT: .cfi_def_cfa_offset 48
+; MIPS2-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; MIPS2-NEXT: .cfi_offset 31, -4
+; MIPS2-NEXT: .cfi_offset 30, -8
+; MIPS2-NEXT: move $fp, $sp
+; MIPS2-NEXT: .cfi_def_cfa_register 30
+; MIPS2-NEXT: addiu $1, $zero, -16
+; MIPS2-NEXT: and $sp, $sp, $1
; MIPS2-NEXT: addu $gp, $2, $25
-; MIPS2-NEXT: lw $1, 60($sp)
-; MIPS2-NEXT: lw $2, 64($sp)
-; MIPS2-NEXT: lw $3, 68($sp)
+; MIPS2-NEXT: lw $1, 68($fp)
+; MIPS2-NEXT: lw $2, 72($fp)
+; MIPS2-NEXT: lw $3, 76($fp)
; MIPS2-NEXT: sw $3, 28($sp)
; MIPS2-NEXT: sw $2, 24($sp)
; MIPS2-NEXT: sw $1, 20($sp)
-; MIPS2-NEXT: lw $1, 56($sp)
+; MIPS2-NEXT: lw $1, 64($fp)
; MIPS2-NEXT: sw $1, 16($sp)
; MIPS2-NEXT: lw $25, %call16(__modti3)($gp)
; MIPS2-NEXT: jalr $25
; MIPS2-NEXT: nop
-; MIPS2-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: move $sp, $fp
+; MIPS2-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS2-NEXT: jr $ra
-; MIPS2-NEXT: addiu $sp, $sp, 40
+; MIPS2-NEXT: addiu $sp, $sp, 48
;
; GP32-LABEL: srem_i128:
; GP32: # %bb.0: # %entry
; GP32-NEXT: lui $2, %hi(_gp_disp)
; GP32-NEXT: addiu $2, $2, %lo(_gp_disp)
-; GP32-NEXT: addiu $sp, $sp, -40
-; GP32-NEXT: .cfi_def_cfa_offset 40
-; GP32-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; GP32-NEXT: addiu $sp, $sp, -48
+; GP32-NEXT: .cfi_def_cfa_offset 48
+; GP32-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; GP32-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; GP32-NEXT: .cfi_offset 31, -4
+; GP32-NEXT: .cfi_offset 30, -8
+; GP32-NEXT: move $fp, $sp
+; GP32-NEXT: .cfi_def_cfa_register 30
+; GP32-NEXT: addiu $1, $zero, -16
+; GP32-NEXT: and $sp, $sp, $1
; GP32-NEXT: addu $gp, $2, $25
-; GP32-NEXT: lw $1, 60($sp)
-; GP32-NEXT: lw $2, 64($sp)
-; GP32-NEXT: lw $3, 68($sp)
+; GP32-NEXT: lw $1, 68($fp)
+; GP32-NEXT: lw $2, 72($fp)
+; GP32-NEXT: lw $3, 76($fp)
; GP32-NEXT: sw $3, 28($sp)
; GP32-NEXT: sw $2, 24($sp)
; GP32-NEXT: sw $1, 20($sp)
-; GP32-NEXT: lw $1, 56($sp)
+; GP32-NEXT: lw $1, 64($fp)
; GP32-NEXT: sw $1, 16($sp)
; GP32-NEXT: lw $25, %call16(__modti3)($gp)
; GP32-NEXT: jalr $25
; GP32-NEXT: nop
-; GP32-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; GP32-NEXT: move $sp, $fp
+; GP32-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; GP32-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; GP32-NEXT: jr $ra
-; GP32-NEXT: addiu $sp, $sp, 40
+; GP32-NEXT: addiu $sp, $sp, 48
;
; GP32R6-LABEL: srem_i128:
; GP32R6: # %bb.0: # %entry
; GP32R6-NEXT: lui $2, %hi(_gp_disp)
; GP32R6-NEXT: addiu $2, $2, %lo(_gp_disp)
-; GP32R6-NEXT: addiu $sp, $sp, -40
-; GP32R6-NEXT: .cfi_def_cfa_offset 40
-; GP32R6-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; GP32R6-NEXT: addiu $sp, $sp, -48
+; GP32R6-NEXT: .cfi_def_cfa_offset 48
+; GP32R6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; GP32R6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; GP32R6-NEXT: .cfi_offset 31, -4
+; GP32R6-NEXT: .cfi_offset 30, -8
+; GP32R6-NEXT: move $fp, $sp
+; GP32R6-NEXT: .cfi_def_cfa_register 30
+; GP32R6-NEXT: addiu $1, $zero, -16
+; GP32R6-NEXT: and $sp, $sp, $1
; GP32R6-NEXT: addu $gp, $2, $25
-; GP32R6-NEXT: lw $1, 60($sp)
-; GP32R6-NEXT: lw $2, 64($sp)
-; GP32R6-NEXT: lw $3, 68($sp)
+; GP32R6-NEXT: lw $1, 68($fp)
+; GP32R6-NEXT: lw $2, 72($fp)
+; GP32R6-NEXT: lw $3, 76($fp)
; GP32R6-NEXT: sw $3, 28($sp)
; GP32R6-NEXT: sw $2, 24($sp)
; GP32R6-NEXT: sw $1, 20($sp)
-; GP32R6-NEXT: lw $1, 56($sp)
+; GP32R6-NEXT: lw $1, 64($fp)
; GP32R6-NEXT: sw $1, 16($sp)
; GP32R6-NEXT: lw $25, %call16(__modti3)($gp)
; GP32R6-NEXT: jalrc $25
-; GP32R6-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; GP32R6-NEXT: move $sp, $fp
+; GP32R6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; GP32R6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; GP32R6-NEXT: jr $ra
-; GP32R6-NEXT: addiu $sp, $sp, 40
+; GP32R6-NEXT: addiu $sp, $sp, 48
;
; MIPS3-LABEL: srem_i128:
; MIPS3: # %bb.0: # %entry
@@ -495,59 +519,75 @@ define signext i128 @srem_i128(i128 signext %a, i128 signext %b) {
; MMR3: # %bb.0: # %entry
; MMR3-NEXT: lui $2, %hi(_gp_disp)
; MMR3-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MMR3-NEXT: addiusp -40
-; MMR3-NEXT: .cfi_def_cfa_offset 40
-; MMR3-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
-; MMR3-NEXT: sw $17, 32($sp) # 4-byte Folded Spill
+; MMR3-NEXT: addiusp -48
+; MMR3-NEXT: .cfi_def_cfa_offset 48
+; MMR3-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $17, 36($sp) # 4-byte Folded Spill
; MMR3-NEXT: .cfi_offset 31, -4
-; MMR3-NEXT: .cfi_offset 17, -8
+; MMR3-NEXT: .cfi_offset 30, -8
+; MMR3-NEXT: .cfi_offset 17, -12
+; MMR3-NEXT: move $fp, $sp
+; MMR3-NEXT: .cfi_def_cfa_register 30
+; MMR3-NEXT: addiu $1, $zero, -16
+; MMR3-NEXT: and $sp, $sp, $1
; MMR3-NEXT: addu $gp, $2, $25
; MMR3-NEXT: move $1, $7
-; MMR3-NEXT: lw $7, 60($sp)
-; MMR3-NEXT: lw $17, 64($sp)
-; MMR3-NEXT: lw $3, 68($sp)
+; MMR3-NEXT: lw $7, 68($fp)
+; MMR3-NEXT: lw $17, 72($fp)
+; MMR3-NEXT: lw $3, 76($fp)
; MMR3-NEXT: move $2, $sp
; MMR3-NEXT: sw16 $3, 28($2)
; MMR3-NEXT: sw16 $17, 24($2)
; MMR3-NEXT: sw16 $7, 20($2)
-; MMR3-NEXT: lw $3, 56($sp)
+; MMR3-NEXT: lw $3, 64($fp)
; MMR3-NEXT: sw16 $3, 16($2)
; MMR3-NEXT: lw $25, %call16(__modti3)($gp)
; MMR3-NEXT: move $7, $1
; MMR3-NEXT: jalr $25
; MMR3-NEXT: nop
-; MMR3-NEXT: lw $17, 32($sp) # 4-byte Folded Reload
-; MMR3-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
-; MMR3-NEXT: addiusp 40
+; MMR3-NEXT: move $sp, $fp
+; MMR3-NEXT: lw $17, 36($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR3-NEXT: addiusp 48
; MMR3-NEXT: jrc $ra
;
; MMR6-LABEL: srem_i128:
; MMR6: # %bb.0: # %entry
; MMR6-NEXT: lui $2, %hi(_gp_disp)
; MMR6-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MMR6-NEXT: addiu $sp, $sp, -40
-; MMR6-NEXT: .cfi_def_cfa_offset 40
-; MMR6-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
-; MMR6-NEXT: sw $17, 32($sp) # 4-byte Folded Spill
+; MMR6-NEXT: addiu $sp, $sp, -48
+; MMR6-NEXT: .cfi_def_cfa_offset 48
+; MMR6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $17, 36($sp) # 4-byte Folded Spill
; MMR6-NEXT: .cfi_offset 31, -4
-; MMR6-NEXT: .cfi_offset 17, -8
+; MMR6-NEXT: .cfi_offset 30, -8
+; MMR6-NEXT: .cfi_offset 17, -12
+; MMR6-NEXT: move $fp, $sp
+; MMR6-NEXT: .cfi_def_cfa_register 30
+; MMR6-NEXT: addiu $1, $zero, -16
+; MMR6-NEXT: and $sp, $sp, $1
; MMR6-NEXT: addu $gp, $2, $25
; MMR6-NEXT: move $1, $7
-; MMR6-NEXT: lw $7, 60($sp)
-; MMR6-NEXT: lw $17, 64($sp)
-; MMR6-NEXT: lw $3, 68($sp)
+; MMR6-NEXT: lw $7, 68($fp)
+; MMR6-NEXT: lw $17, 72($fp)
+; MMR6-NEXT: lw $3, 76($fp)
; MMR6-NEXT: move $2, $sp
; MMR6-NEXT: sw16 $3, 28($2)
; MMR6-NEXT: sw16 $17, 24($2)
; MMR6-NEXT: sw16 $7, 20($2)
-; MMR6-NEXT: lw $3, 56($sp)
+; MMR6-NEXT: lw $3, 64($fp)
; MMR6-NEXT: sw16 $3, 16($2)
; MMR6-NEXT: lw $25, %call16(__modti3)($gp)
; MMR6-NEXT: move $7, $1
; MMR6-NEXT: jalr $25
-; MMR6-NEXT: lw $17, 32($sp) # 4-byte Folded Reload
-; MMR6-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
-; MMR6-NEXT: addiu $sp, $sp, 40
+; MMR6-NEXT: move $sp, $fp
+; MMR6-NEXT: lw $17, 36($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR6-NEXT: addiu $sp, $sp, 48
; MMR6-NEXT: jrc $ra
entry:
%r = srem i128 %a, %b
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/sub.ll b/llvm/test/CodeGen/Mips/llvm-ir/sub.ll
index dd5e6e957245d..0af0e14961d4c 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/sub.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/sub.ll
@@ -113,8 +113,8 @@ define signext i128 @sub_i128(i128 signext %a, i128 signext %b) {
entry:
; ALL-LABEL: sub_i128:
-; PRE4: lw $[[T0:[0-9]+]], 24($sp)
-; PRE4: lw $[[T1:[0-9]+]], 28($sp)
+; PRE4: lw $[[T0:[0-9]+]], 40($fp)
+; PRE4: lw $[[T1:[0-9]+]], 44($fp)
; PRE4: sltu $[[T2:[0-9]+]], $7, $[[T1]]
; PRE4: xor $[[T3:[0-9]+]], $6, $[[T0]]
; PRE4: sltiu $[[T4:[0-9]+]], $[[T3]], 1
@@ -122,12 +122,12 @@ entry:
; PRE4: move $[[T5:[0-9]+]], $[[T2]]
; PRE4: sltu $[[T5]], $6, $[[T0]]
-; PRE4: lw $[[T6:[0-9]+]], 20($sp)
+; PRE4: lw $[[T6:[0-9]+]], 36($fp)
; PRE4: subu $[[T7:[0-9]+]], $5, $[[T6]]
; PRE4: subu $[[T8:[0-9]+]], $[[T7]], $[[T5]]
; PRE4: sltu $[[T9:[0-9]+]], $[[T7]], $[[T5]]
; PRE4: sltu $[[T10:[0-9]+]], $5, $[[T6]]
-; PRE4: lw $[[T11:[0-9]+]], 16($sp)
+; PRE4: lw $[[T11:[0-9]+]], 32($fp)
; PRE4: subu $[[T12:[0-9]+]], $4, $[[T11]]
; PRE4: subu $[[T13:[0-9]+]], $[[T12]], $[[T10]]
; PRE4: subu $[[T14:[0-9]+]], $[[T13]], $[[T9]]
@@ -135,24 +135,24 @@ entry:
; PRE4: subu $[[T16:[0-9]+]], $[[T15]], $[[T2]]
; PRE4: subu $5, $7, $[[T1]]
-; MMR3: lw $[[T1:[0-9]+]], 48($sp)
+; MMR3: lw $[[T1:[0-9]+]], 56($fp)
; MMR3: sltu $[[T2:[0-9]+]], $6, $[[T1]]
; MMR3: xor $[[T3:[0-9]+]], $6, $[[T1]]
-; MMR3: lw $[[T4:[0-9]+]], 52($sp)
+; MMR3: lw $[[T4:[0-9]+]], 60($fp)
; MMR3: sltu $[[T5:[0-9]+]], $7, $[[T4]]
; MMR3: movz $[[T6:[0-9]+]], $[[T5]], $[[T3]]
-; MMR3: lw $[[T7:[0-8]+]], 44($sp)
+; MMR3: lw $[[T7:[0-9]+]], 52($fp)
; MMR3: subu16 $[[T8:[0-9]+]], $5, $[[T7]]
; MMR3: subu16 $[[T9:[0-9]+]], $[[T8]], $[[T6]]
-; MMR3: sltu $[[T10:[0-9]+]], $[[T8]], $[[T2]]
+; MMR3: sltu $[[T10:[0-9]+]], $[[T8]], $[[T6]]
; MMR3: sltu $[[T11:[0-9]+]], $5, $[[T7]]
-; MMR3: lw $[[T12:[0-9]+]], 40($sp)
+; MMR3: lw $[[T12:[0-9]+]], 48($fp)
; MMR3: lw $[[T13:[0-9]+]], 12($sp)
; MMR3: subu16 $[[T14:[0-9]+]], $[[T13]], $[[T12]]
; MMR3: subu16 $[[T15:[0-9]+]], $[[T14]], $[[T11]]
; MMR3: subu16 $[[T16:[0-9]+]], $[[T15]], $[[T10]]
; MMR3: subu16 $[[T17:[0-9]+]], $6, $[[T1]]
-; MMR3: subu16 $[[T18:[0-9]+]], $[[T17]], $7
+; MMR3: subu16 $[[T18:[0-9]+]], $[[T17]], $[[T5]]
; MMR3: lw $[[T19:[0-9]+]], 8($sp)
; MMR3: lw $[[T20:[0-9]+]], 0($sp)
; MMR3: subu16 $5, $[[T19]], $[[T20]]
@@ -161,27 +161,27 @@ entry:
; MMR6: sw $7, 8($sp)
; MMR6: move $[[T1:[0-9]+]], $5
; MMR6: sw $4, 12($sp)
-; MMR6: lw $[[T2:[0-9]+]], 48($sp)
+; MMR6: lw $[[T2:[0-9]+]], 56($fp)
; MMR6: sltu $[[T3:[0-9]+]], $6, $[[T2]]
; MMR6: xor $[[T4:[0-9]+]], $6, $[[T2]]
; MMR6: sltiu $[[T5:[0-9]+]], $[[T4]], 1
; MMR6: seleqz $[[T6:[0-9]+]], $[[T3]], $[[T5]]
-; MMR6: lw $[[T7:[0-9]+]], 52($sp)
+; MMR6: lw $[[T7:[0-9]+]], 60($fp)
; MMR6: sltu $[[T8:[0-9]+]], $[[T0]], $[[T7]]
; MMR6: selnez $[[T9:[0-9]+]], $[[T8]], $[[T5]]
; MMR6: or $[[T10:[0-9]+]], $[[T9]], $[[T6]]
-; MMR6: lw $[[T11:[0-9]+]], 44($sp)
+; MMR6: lw $[[T11:[0-9]+]], 52($fp)
; MMR6: subu16 $[[T12:[0-9]+]], $[[T1]], $[[T11]]
-; MMR6: subu16 $[[T13:[0-9]+]], $[[T12]], $[[T7]]
-; MMR6: sltu $[[T16:[0-9]+]], $[[T12]], $[[T7]]
+; MMR6: subu16 $[[T13:[0-9]+]], $[[T12]], $[[T10]]
+; MMR6: sltu $[[T16:[0-9]+]], $[[T12]], $[[T10]]
; MMR6: sltu $[[T17:[0-9]+]], $[[T1]], $[[T11]]
-; MMR6: lw $[[T18:[0-9]+]], 40($sp)
+; MMR6: lw $[[T18:[0-9]+]], 48($fp)
; MMR6: lw $[[T19:[0-9]+]], 12($sp)
; MMR6: subu16 $[[T20:[0-9]+]], $[[T19]], $[[T18]]
; MMR6: subu16 $[[T21:[0-9]+]], $[[T20]], $[[T17]]
; MMR6: subu16 $[[T22:[0-9]+]], $[[T21]], $[[T16]]
; MMR6: subu16 $[[T23:[0-9]+]], $6, $[[T2]]
-; MMR6: subu16 $4, $[[T23]], $5
+; MMR6: subu16 $4, $[[T23]], $[[T8]]
; MMR6: lw $[[T24:[0-9]+]], 8($sp)
; MMR6: lw $[[T25:[0-9]+]], 0($sp)
; MMR6: subu16 $5, $[[T24]], $[[T25]]
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/udiv.ll b/llvm/test/CodeGen/Mips/llvm-ir/udiv.ll
index 9451f1e9be096..5e8f53555c434 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/udiv.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/udiv.ll
@@ -368,72 +368,96 @@ define signext i128 @udiv_i128(i128 signext %a, i128 signext %b) {
; MIPS2: # %bb.0: # %entry
; MIPS2-NEXT: lui $2, %hi(_gp_disp)
; MIPS2-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MIPS2-NEXT: addiu $sp, $sp, -40
-; MIPS2-NEXT: .cfi_def_cfa_offset 40
-; MIPS2-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: addiu $sp, $sp, -48
+; MIPS2-NEXT: .cfi_def_cfa_offset 48
+; MIPS2-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; MIPS2-NEXT: .cfi_offset 31, -4
+; MIPS2-NEXT: .cfi_offset 30, -8
+; MIPS2-NEXT: move $fp, $sp
+; MIPS2-NEXT: .cfi_def_cfa_register 30
+; MIPS2-NEXT: addiu $1, $zero, -16
+; MIPS2-NEXT: and $sp, $sp, $1
; MIPS2-NEXT: addu $gp, $2, $25
-; MIPS2-NEXT: lw $1, 60($sp)
-; MIPS2-NEXT: lw $2, 64($sp)
-; MIPS2-NEXT: lw $3, 68($sp)
+; MIPS2-NEXT: lw $1, 68($fp)
+; MIPS2-NEXT: lw $2, 72($fp)
+; MIPS2-NEXT: lw $3, 76($fp)
; MIPS2-NEXT: sw $3, 28($sp)
; MIPS2-NEXT: sw $2, 24($sp)
; MIPS2-NEXT: sw $1, 20($sp)
-; MIPS2-NEXT: lw $1, 56($sp)
+; MIPS2-NEXT: lw $1, 64($fp)
; MIPS2-NEXT: sw $1, 16($sp)
; MIPS2-NEXT: lw $25, %call16(__udivti3)($gp)
; MIPS2-NEXT: jalr $25
; MIPS2-NEXT: nop
-; MIPS2-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: move $sp, $fp
+; MIPS2-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS2-NEXT: jr $ra
-; MIPS2-NEXT: addiu $sp, $sp, 40
+; MIPS2-NEXT: addiu $sp, $sp, 48
;
; GP32-LABEL: udiv_i128:
; GP32: # %bb.0: # %entry
; GP32-NEXT: lui $2, %hi(_gp_disp)
; GP32-NEXT: addiu $2, $2, %lo(_gp_disp)
-; GP32-NEXT: addiu $sp, $sp, -40
-; GP32-NEXT: .cfi_def_cfa_offset 40
-; GP32-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; GP32-NEXT: addiu $sp, $sp, -48
+; GP32-NEXT: .cfi_def_cfa_offset 48
+; GP32-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; GP32-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; GP32-NEXT: .cfi_offset 31, -4
+; GP32-NEXT: .cfi_offset 30, -8
+; GP32-NEXT: move $fp, $sp
+; GP32-NEXT: .cfi_def_cfa_register 30
+; GP32-NEXT: addiu $1, $zero, -16
+; GP32-NEXT: and $sp, $sp, $1
; GP32-NEXT: addu $gp, $2, $25
-; GP32-NEXT: lw $1, 60($sp)
-; GP32-NEXT: lw $2, 64($sp)
-; GP32-NEXT: lw $3, 68($sp)
+; GP32-NEXT: lw $1, 68($fp)
+; GP32-NEXT: lw $2, 72($fp)
+; GP32-NEXT: lw $3, 76($fp)
; GP32-NEXT: sw $3, 28($sp)
; GP32-NEXT: sw $2, 24($sp)
; GP32-NEXT: sw $1, 20($sp)
-; GP32-NEXT: lw $1, 56($sp)
+; GP32-NEXT: lw $1, 64($fp)
; GP32-NEXT: sw $1, 16($sp)
; GP32-NEXT: lw $25, %call16(__udivti3)($gp)
; GP32-NEXT: jalr $25
; GP32-NEXT: nop
-; GP32-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; GP32-NEXT: move $sp, $fp
+; GP32-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; GP32-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; GP32-NEXT: jr $ra
-; GP32-NEXT: addiu $sp, $sp, 40
+; GP32-NEXT: addiu $sp, $sp, 48
;
; GP32R6-LABEL: udiv_i128:
; GP32R6: # %bb.0: # %entry
; GP32R6-NEXT: lui $2, %hi(_gp_disp)
; GP32R6-NEXT: addiu $2, $2, %lo(_gp_disp)
-; GP32R6-NEXT: addiu $sp, $sp, -40
-; GP32R6-NEXT: .cfi_def_cfa_offset 40
-; GP32R6-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; GP32R6-NEXT: addiu $sp, $sp, -48
+; GP32R6-NEXT: .cfi_def_cfa_offset 48
+; GP32R6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; GP32R6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; GP32R6-NEXT: .cfi_offset 31, -4
+; GP32R6-NEXT: .cfi_offset 30, -8
+; GP32R6-NEXT: move $fp, $sp
+; GP32R6-NEXT: .cfi_def_cfa_register 30
+; GP32R6-NEXT: addiu $1, $zero, -16
+; GP32R6-NEXT: and $sp, $sp, $1
; GP32R6-NEXT: addu $gp, $2, $25
-; GP32R6-NEXT: lw $1, 60($sp)
-; GP32R6-NEXT: lw $2, 64($sp)
-; GP32R6-NEXT: lw $3, 68($sp)
+; GP32R6-NEXT: lw $1, 68($fp)
+; GP32R6-NEXT: lw $2, 72($fp)
+; GP32R6-NEXT: lw $3, 76($fp)
; GP32R6-NEXT: sw $3, 28($sp)
; GP32R6-NEXT: sw $2, 24($sp)
; GP32R6-NEXT: sw $1, 20($sp)
-; GP32R6-NEXT: lw $1, 56($sp)
+; GP32R6-NEXT: lw $1, 64($fp)
; GP32R6-NEXT: sw $1, 16($sp)
; GP32R6-NEXT: lw $25, %call16(__udivti3)($gp)
; GP32R6-NEXT: jalrc $25
-; GP32R6-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; GP32R6-NEXT: move $sp, $fp
+; GP32R6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; GP32R6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; GP32R6-NEXT: jr $ra
-; GP32R6-NEXT: addiu $sp, $sp, 40
+; GP32R6-NEXT: addiu $sp, $sp, 48
;
; MIPS3-LABEL: udiv_i128:
; MIPS3: # %bb.0: # %entry
@@ -495,59 +519,75 @@ define signext i128 @udiv_i128(i128 signext %a, i128 signext %b) {
; MMR3: # %bb.0: # %entry
; MMR3-NEXT: lui $2, %hi(_gp_disp)
; MMR3-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MMR3-NEXT: addiusp -40
-; MMR3-NEXT: .cfi_def_cfa_offset 40
-; MMR3-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
-; MMR3-NEXT: sw $17, 32($sp) # 4-byte Folded Spill
+; MMR3-NEXT: addiusp -48
+; MMR3-NEXT: .cfi_def_cfa_offset 48
+; MMR3-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $17, 36($sp) # 4-byte Folded Spill
; MMR3-NEXT: .cfi_offset 31, -4
-; MMR3-NEXT: .cfi_offset 17, -8
+; MMR3-NEXT: .cfi_offset 30, -8
+; MMR3-NEXT: .cfi_offset 17, -12
+; MMR3-NEXT: move $fp, $sp
+; MMR3-NEXT: .cfi_def_cfa_register 30
+; MMR3-NEXT: addiu $1, $zero, -16
+; MMR3-NEXT: and $sp, $sp, $1
; MMR3-NEXT: addu $gp, $2, $25
; MMR3-NEXT: move $1, $7
-; MMR3-NEXT: lw $7, 60($sp)
-; MMR3-NEXT: lw $17, 64($sp)
-; MMR3-NEXT: lw $3, 68($sp)
+; MMR3-NEXT: lw $7, 68($fp)
+; MMR3-NEXT: lw $17, 72($fp)
+; MMR3-NEXT: lw $3, 76($fp)
; MMR3-NEXT: move $2, $sp
; MMR3-NEXT: sw16 $3, 28($2)
; MMR3-NEXT: sw16 $17, 24($2)
; MMR3-NEXT: sw16 $7, 20($2)
-; MMR3-NEXT: lw $3, 56($sp)
+; MMR3-NEXT: lw $3, 64($fp)
; MMR3-NEXT: sw16 $3, 16($2)
; MMR3-NEXT: lw $25, %call16(__udivti3)($gp)
; MMR3-NEXT: move $7, $1
; MMR3-NEXT: jalr $25
; MMR3-NEXT: nop
-; MMR3-NEXT: lw $17, 32($sp) # 4-byte Folded Reload
-; MMR3-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
-; MMR3-NEXT: addiusp 40
+; MMR3-NEXT: move $sp, $fp
+; MMR3-NEXT: lw $17, 36($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR3-NEXT: addiusp 48
; MMR3-NEXT: jrc $ra
;
; MMR6-LABEL: udiv_i128:
; MMR6: # %bb.0: # %entry
; MMR6-NEXT: lui $2, %hi(_gp_disp)
; MMR6-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MMR6-NEXT: addiu $sp, $sp, -40
-; MMR6-NEXT: .cfi_def_cfa_offset 40
-; MMR6-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
-; MMR6-NEXT: sw $17, 32($sp) # 4-byte Folded Spill
+; MMR6-NEXT: addiu $sp, $sp, -48
+; MMR6-NEXT: .cfi_def_cfa_offset 48
+; MMR6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $17, 36($sp) # 4-byte Folded Spill
; MMR6-NEXT: .cfi_offset 31, -4
-; MMR6-NEXT: .cfi_offset 17, -8
+; MMR6-NEXT: .cfi_offset 30, -8
+; MMR6-NEXT: .cfi_offset 17, -12
+; MMR6-NEXT: move $fp, $sp
+; MMR6-NEXT: .cfi_def_cfa_register 30
+; MMR6-NEXT: addiu $1, $zero, -16
+; MMR6-NEXT: and $sp, $sp, $1
; MMR6-NEXT: addu $gp, $2, $25
; MMR6-NEXT: move $1, $7
-; MMR6-NEXT: lw $7, 60($sp)
-; MMR6-NEXT: lw $17, 64($sp)
-; MMR6-NEXT: lw $3, 68($sp)
+; MMR6-NEXT: lw $7, 68($fp)
+; MMR6-NEXT: lw $17, 72($fp)
+; MMR6-NEXT: lw $3, 76($fp)
; MMR6-NEXT: move $2, $sp
; MMR6-NEXT: sw16 $3, 28($2)
; MMR6-NEXT: sw16 $17, 24($2)
; MMR6-NEXT: sw16 $7, 20($2)
-; MMR6-NEXT: lw $3, 56($sp)
+; MMR6-NEXT: lw $3, 64($fp)
; MMR6-NEXT: sw16 $3, 16($2)
; MMR6-NEXT: lw $25, %call16(__udivti3)($gp)
; MMR6-NEXT: move $7, $1
; MMR6-NEXT: jalr $25
-; MMR6-NEXT: lw $17, 32($sp) # 4-byte Folded Reload
-; MMR6-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
-; MMR6-NEXT: addiu $sp, $sp, 40
+; MMR6-NEXT: move $sp, $fp
+; MMR6-NEXT: lw $17, 36($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR6-NEXT: addiu $sp, $sp, 48
; MMR6-NEXT: jrc $ra
entry:
%r = udiv i128 %a, %b
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/urem.ll b/llvm/test/CodeGen/Mips/llvm-ir/urem.ll
index 7bd8df9e48c75..e0829fb696ded 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/urem.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/urem.ll
@@ -460,72 +460,96 @@ define signext i128 @urem_i128(i128 signext %a, i128 signext %b) {
; MIPS2: # %bb.0: # %entry
; MIPS2-NEXT: lui $2, %hi(_gp_disp)
; MIPS2-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MIPS2-NEXT: addiu $sp, $sp, -40
-; MIPS2-NEXT: .cfi_def_cfa_offset 40
-; MIPS2-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: addiu $sp, $sp, -48
+; MIPS2-NEXT: .cfi_def_cfa_offset 48
+; MIPS2-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MIPS2-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; MIPS2-NEXT: .cfi_offset 31, -4
+; MIPS2-NEXT: .cfi_offset 30, -8
+; MIPS2-NEXT: move $fp, $sp
+; MIPS2-NEXT: .cfi_def_cfa_register 30
+; MIPS2-NEXT: addiu $1, $zero, -16
+; MIPS2-NEXT: and $sp, $sp, $1
; MIPS2-NEXT: addu $gp, $2, $25
-; MIPS2-NEXT: lw $1, 60($sp)
-; MIPS2-NEXT: lw $2, 64($sp)
-; MIPS2-NEXT: lw $3, 68($sp)
+; MIPS2-NEXT: lw $1, 68($fp)
+; MIPS2-NEXT: lw $2, 72($fp)
+; MIPS2-NEXT: lw $3, 76($fp)
; MIPS2-NEXT: sw $3, 28($sp)
; MIPS2-NEXT: sw $2, 24($sp)
; MIPS2-NEXT: sw $1, 20($sp)
-; MIPS2-NEXT: lw $1, 56($sp)
+; MIPS2-NEXT: lw $1, 64($fp)
; MIPS2-NEXT: sw $1, 16($sp)
; MIPS2-NEXT: lw $25, %call16(__umodti3)($gp)
; MIPS2-NEXT: jalr $25
; MIPS2-NEXT: nop
-; MIPS2-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: move $sp, $fp
+; MIPS2-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MIPS2-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; MIPS2-NEXT: jr $ra
-; MIPS2-NEXT: addiu $sp, $sp, 40
+; MIPS2-NEXT: addiu $sp, $sp, 48
;
; GP32-LABEL: urem_i128:
; GP32: # %bb.0: # %entry
; GP32-NEXT: lui $2, %hi(_gp_disp)
; GP32-NEXT: addiu $2, $2, %lo(_gp_disp)
-; GP32-NEXT: addiu $sp, $sp, -40
-; GP32-NEXT: .cfi_def_cfa_offset 40
-; GP32-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; GP32-NEXT: addiu $sp, $sp, -48
+; GP32-NEXT: .cfi_def_cfa_offset 48
+; GP32-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; GP32-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; GP32-NEXT: .cfi_offset 31, -4
+; GP32-NEXT: .cfi_offset 30, -8
+; GP32-NEXT: move $fp, $sp
+; GP32-NEXT: .cfi_def_cfa_register 30
+; GP32-NEXT: addiu $1, $zero, -16
+; GP32-NEXT: and $sp, $sp, $1
; GP32-NEXT: addu $gp, $2, $25
-; GP32-NEXT: lw $1, 60($sp)
-; GP32-NEXT: lw $2, 64($sp)
-; GP32-NEXT: lw $3, 68($sp)
+; GP32-NEXT: lw $1, 68($fp)
+; GP32-NEXT: lw $2, 72($fp)
+; GP32-NEXT: lw $3, 76($fp)
; GP32-NEXT: sw $3, 28($sp)
; GP32-NEXT: sw $2, 24($sp)
; GP32-NEXT: sw $1, 20($sp)
-; GP32-NEXT: lw $1, 56($sp)
+; GP32-NEXT: lw $1, 64($fp)
; GP32-NEXT: sw $1, 16($sp)
; GP32-NEXT: lw $25, %call16(__umodti3)($gp)
; GP32-NEXT: jalr $25
; GP32-NEXT: nop
-; GP32-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; GP32-NEXT: move $sp, $fp
+; GP32-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; GP32-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; GP32-NEXT: jr $ra
-; GP32-NEXT: addiu $sp, $sp, 40
+; GP32-NEXT: addiu $sp, $sp, 48
;
; GP32R6-LABEL: urem_i128:
; GP32R6: # %bb.0: # %entry
; GP32R6-NEXT: lui $2, %hi(_gp_disp)
; GP32R6-NEXT: addiu $2, $2, %lo(_gp_disp)
-; GP32R6-NEXT: addiu $sp, $sp, -40
-; GP32R6-NEXT: .cfi_def_cfa_offset 40
-; GP32R6-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
+; GP32R6-NEXT: addiu $sp, $sp, -48
+; GP32R6-NEXT: .cfi_def_cfa_offset 48
+; GP32R6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; GP32R6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
; GP32R6-NEXT: .cfi_offset 31, -4
+; GP32R6-NEXT: .cfi_offset 30, -8
+; GP32R6-NEXT: move $fp, $sp
+; GP32R6-NEXT: .cfi_def_cfa_register 30
+; GP32R6-NEXT: addiu $1, $zero, -16
+; GP32R6-NEXT: and $sp, $sp, $1
; GP32R6-NEXT: addu $gp, $2, $25
-; GP32R6-NEXT: lw $1, 60($sp)
-; GP32R6-NEXT: lw $2, 64($sp)
-; GP32R6-NEXT: lw $3, 68($sp)
+; GP32R6-NEXT: lw $1, 68($fp)
+; GP32R6-NEXT: lw $2, 72($fp)
+; GP32R6-NEXT: lw $3, 76($fp)
; GP32R6-NEXT: sw $3, 28($sp)
; GP32R6-NEXT: sw $2, 24($sp)
; GP32R6-NEXT: sw $1, 20($sp)
-; GP32R6-NEXT: lw $1, 56($sp)
+; GP32R6-NEXT: lw $1, 64($fp)
; GP32R6-NEXT: sw $1, 16($sp)
; GP32R6-NEXT: lw $25, %call16(__umodti3)($gp)
; GP32R6-NEXT: jalrc $25
-; GP32R6-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
+; GP32R6-NEXT: move $sp, $fp
+; GP32R6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; GP32R6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
; GP32R6-NEXT: jr $ra
-; GP32R6-NEXT: addiu $sp, $sp, 40
+; GP32R6-NEXT: addiu $sp, $sp, 48
;
; MIPS3-LABEL: urem_i128:
; MIPS3: # %bb.0: # %entry
@@ -587,59 +611,75 @@ define signext i128 @urem_i128(i128 signext %a, i128 signext %b) {
; MMR3: # %bb.0: # %entry
; MMR3-NEXT: lui $2, %hi(_gp_disp)
; MMR3-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MMR3-NEXT: addiusp -40
-; MMR3-NEXT: .cfi_def_cfa_offset 40
-; MMR3-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
-; MMR3-NEXT: sw $17, 32($sp) # 4-byte Folded Spill
+; MMR3-NEXT: addiusp -48
+; MMR3-NEXT: .cfi_def_cfa_offset 48
+; MMR3-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MMR3-NEXT: sw $17, 36($sp) # 4-byte Folded Spill
; MMR3-NEXT: .cfi_offset 31, -4
-; MMR3-NEXT: .cfi_offset 17, -8
+; MMR3-NEXT: .cfi_offset 30, -8
+; MMR3-NEXT: .cfi_offset 17, -12
+; MMR3-NEXT: move $fp, $sp
+; MMR3-NEXT: .cfi_def_cfa_register 30
+; MMR3-NEXT: addiu $1, $zero, -16
+; MMR3-NEXT: and $sp, $sp, $1
; MMR3-NEXT: addu $gp, $2, $25
; MMR3-NEXT: move $1, $7
-; MMR3-NEXT: lw $7, 60($sp)
-; MMR3-NEXT: lw $17, 64($sp)
-; MMR3-NEXT: lw $3, 68($sp)
+; MMR3-NEXT: lw $7, 68($fp)
+; MMR3-NEXT: lw $17, 72($fp)
+; MMR3-NEXT: lw $3, 76($fp)
; MMR3-NEXT: move $2, $sp
; MMR3-NEXT: sw16 $3, 28($2)
; MMR3-NEXT: sw16 $17, 24($2)
; MMR3-NEXT: sw16 $7, 20($2)
-; MMR3-NEXT: lw $3, 56($sp)
+; MMR3-NEXT: lw $3, 64($fp)
; MMR3-NEXT: sw16 $3, 16($2)
; MMR3-NEXT: lw $25, %call16(__umodti3)($gp)
; MMR3-NEXT: move $7, $1
; MMR3-NEXT: jalr $25
; MMR3-NEXT: nop
-; MMR3-NEXT: lw $17, 32($sp) # 4-byte Folded Reload
-; MMR3-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
-; MMR3-NEXT: addiusp 40
+; MMR3-NEXT: move $sp, $fp
+; MMR3-NEXT: lw $17, 36($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR3-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR3-NEXT: addiusp 48
; MMR3-NEXT: jrc $ra
;
; MMR6-LABEL: urem_i128:
; MMR6: # %bb.0: # %entry
; MMR6-NEXT: lui $2, %hi(_gp_disp)
; MMR6-NEXT: addiu $2, $2, %lo(_gp_disp)
-; MMR6-NEXT: addiu $sp, $sp, -40
-; MMR6-NEXT: .cfi_def_cfa_offset 40
-; MMR6-NEXT: sw $ra, 36($sp) # 4-byte Folded Spill
-; MMR6-NEXT: sw $17, 32($sp) # 4-byte Folded Spill
+; MMR6-NEXT: addiu $sp, $sp, -48
+; MMR6-NEXT: .cfi_def_cfa_offset 48
+; MMR6-NEXT: sw $ra, 44($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $fp, 40($sp) # 4-byte Folded Spill
+; MMR6-NEXT: sw $17, 36($sp) # 4-byte Folded Spill
; MMR6-NEXT: .cfi_offset 31, -4
-; MMR6-NEXT: .cfi_offset 17, -8
+; MMR6-NEXT: .cfi_offset 30, -8
+; MMR6-NEXT: .cfi_offset 17, -12
+; MMR6-NEXT: move $fp, $sp
+; MMR6-NEXT: .cfi_def_cfa_register 30
+; MMR6-NEXT: addiu $1, $zero, -16
+; MMR6-NEXT: and $sp, $sp, $1
; MMR6-NEXT: addu $gp, $2, $25
; MMR6-NEXT: move $1, $7
-; MMR6-NEXT: lw $7, 60($sp)
-; MMR6-NEXT: lw $17, 64($sp)
-; MMR6-NEXT: lw $3, 68($sp)
+; MMR6-NEXT: lw $7, 68($fp)
+; MMR6-NEXT: lw $17, 72($fp)
+; MMR6-NEXT: lw $3, 76($fp)
; MMR6-NEXT: move $2, $sp
; MMR6-NEXT: sw16 $3, 28($2)
; MMR6-NEXT: sw16 $17, 24($2)
; MMR6-NEXT: sw16 $7, 20($2)
-; MMR6-NEXT: lw $3, 56($sp)
+; MMR6-NEXT: lw $3, 64($fp)
; MMR6-NEXT: sw16 $3, 16($2)
; MMR6-NEXT: lw $25, %call16(__umodti3)($gp)
; MMR6-NEXT: move $7, $1
; MMR6-NEXT: jalr $25
-; MMR6-NEXT: lw $17, 32($sp) # 4-byte Folded Reload
-; MMR6-NEXT: lw $ra, 36($sp) # 4-byte Folded Reload
-; MMR6-NEXT: addiu $sp, $sp, 40
+; MMR6-NEXT: move $sp, $fp
+; MMR6-NEXT: lw $17, 36($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $fp, 40($sp) # 4-byte Folded Reload
+; MMR6-NEXT: lw $ra, 44($sp) # 4-byte Folded Reload
+; MMR6-NEXT: addiu $sp, $sp, 48
; MMR6-NEXT: jrc $ra
entry:
%r = urem i128 %a, %b
diff --git a/llvm/test/CodeGen/Mips/llvm-ir/xor.ll b/llvm/test/CodeGen/Mips/llvm-ir/xor.ll
index ec9a204b2b431..0364afdfe2efb 100644
--- a/llvm/test/CodeGen/Mips/llvm-ir/xor.ll
+++ b/llvm/test/CodeGen/Mips/llvm-ir/xor.ll
@@ -1,3 +1,4 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 6
; RUN: llc < %s -mtriple=mips-unknown-linux-gnu -mcpu=mips2 | FileCheck %s -check-prefix=MIPS
; RUN: llc < %s -mtriple=mips-unknown-linux-gnu -mcpu=mips32 | FileCheck %s -check-prefix=MIPS
; RUN: llc < %s -mtriple=mips-unknown-linux-gnu -mcpu=mips32r2 | FileCheck %s \
@@ -275,39 +276,81 @@ entry:
define signext i128 @xor_i128(i128 signext %a, i128 signext %b) {
; MIPS-LABEL: xor_i128:
; MIPS: # %bb.0: # %entry
-; MIPS-NEXT: lw $1, 20($sp)
-; MIPS-NEXT: lw $2, 16($sp)
+; MIPS-NEXT: addiu $sp, $sp, -16
+; MIPS-NEXT: .cfi_def_cfa_offset 16
+; MIPS-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MIPS-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MIPS-NEXT: .cfi_offset 31, -4
+; MIPS-NEXT: .cfi_offset 30, -8
+; MIPS-NEXT: move $fp, $sp
+; MIPS-NEXT: .cfi_def_cfa_register 30
+; MIPS-NEXT: addiu $1, $zero, -16
+; MIPS-NEXT: and $sp, $sp, $1
+; MIPS-NEXT: lw $1, 36($fp)
+; MIPS-NEXT: lw $2, 32($fp)
; MIPS-NEXT: xor $2, $4, $2
; MIPS-NEXT: xor $3, $5, $1
-; MIPS-NEXT: lw $1, 24($sp)
+; MIPS-NEXT: lw $1, 40($fp)
; MIPS-NEXT: xor $4, $6, $1
-; MIPS-NEXT: lw $1, 28($sp)
-; MIPS-NEXT: jr $ra
+; MIPS-NEXT: lw $1, 44($fp)
; MIPS-NEXT: xor $5, $7, $1
+; MIPS-NEXT: move $sp, $fp
+; MIPS-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MIPS-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MIPS-NEXT: jr $ra
+; MIPS-NEXT: addiu $sp, $sp, 16
;
; MIPS32R2-LABEL: xor_i128:
; MIPS32R2: # %bb.0: # %entry
-; MIPS32R2-NEXT: lw $1, 20($sp)
-; MIPS32R2-NEXT: lw $2, 16($sp)
+; MIPS32R2-NEXT: addiu $sp, $sp, -16
+; MIPS32R2-NEXT: .cfi_def_cfa_offset 16
+; MIPS32R2-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MIPS32R2-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MIPS32R2-NEXT: .cfi_offset 31, -4
+; MIPS32R2-NEXT: .cfi_offset 30, -8
+; MIPS32R2-NEXT: move $fp, $sp
+; MIPS32R2-NEXT: .cfi_def_cfa_register 30
+; MIPS32R2-NEXT: addiu $1, $zero, -16
+; MIPS32R2-NEXT: and $sp, $sp, $1
+; MIPS32R2-NEXT: lw $1, 36($fp)
+; MIPS32R2-NEXT: lw $2, 32($fp)
; MIPS32R2-NEXT: xor $2, $4, $2
; MIPS32R2-NEXT: xor $3, $5, $1
-; MIPS32R2-NEXT: lw $1, 24($sp)
+; MIPS32R2-NEXT: lw $1, 40($fp)
; MIPS32R2-NEXT: xor $4, $6, $1
-; MIPS32R2-NEXT: lw $1, 28($sp)
-; MIPS32R2-NEXT: jr $ra
+; MIPS32R2-NEXT: lw $1, 44($fp)
; MIPS32R2-NEXT: xor $5, $7, $1
+; MIPS32R2-NEXT: move $sp, $fp
+; MIPS32R2-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MIPS32R2-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MIPS32R2-NEXT: jr $ra
+; MIPS32R2-NEXT: addiu $sp, $sp, 16
;
; MIPS32R6-LABEL: xor_i128:
; MIPS32R6: # %bb.0: # %entry
-; MIPS32R6-NEXT: lw $1, 20($sp)
-; MIPS32R6-NEXT: lw $2, 16($sp)
+; MIPS32R6-NEXT: addiu $sp, $sp, -16
+; MIPS32R6-NEXT: .cfi_def_cfa_offset 16
+; MIPS32R6-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MIPS32R6-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MIPS32R6-NEXT: .cfi_offset 31, -4
+; MIPS32R6-NEXT: .cfi_offset 30, -8
+; MIPS32R6-NEXT: move $fp, $sp
+; MIPS32R6-NEXT: .cfi_def_cfa_register 30
+; MIPS32R6-NEXT: addiu $1, $zero, -16
+; MIPS32R6-NEXT: and $sp, $sp, $1
+; MIPS32R6-NEXT: lw $1, 36($fp)
+; MIPS32R6-NEXT: lw $2, 32($fp)
; MIPS32R6-NEXT: xor $2, $4, $2
; MIPS32R6-NEXT: xor $3, $5, $1
-; MIPS32R6-NEXT: lw $1, 24($sp)
+; MIPS32R6-NEXT: lw $1, 40($fp)
; MIPS32R6-NEXT: xor $4, $6, $1
-; MIPS32R6-NEXT: lw $1, 28($sp)
-; MIPS32R6-NEXT: jr $ra
+; MIPS32R6-NEXT: lw $1, 44($fp)
; MIPS32R6-NEXT: xor $5, $7, $1
+; MIPS32R6-NEXT: move $sp, $fp
+; MIPS32R6-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MIPS32R6-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MIPS32R6-NEXT: jr $ra
+; MIPS32R6-NEXT: addiu $sp, $sp, 16
;
; MIPS64-LABEL: xor_i128:
; MIPS64: # %bb.0: # %entry
@@ -329,25 +372,53 @@ define signext i128 @xor_i128(i128 signext %a, i128 signext %b) {
;
; MM32R3-LABEL: xor_i128:
; MM32R3: # %bb.0: # %entry
-; MM32R3-NEXT: lwp $2, 16($sp)
+; MM32R3-NEXT: addiusp -16
+; MM32R3-NEXT: .cfi_def_cfa_offset 16
+; MM32R3-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MM32R3-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MM32R3-NEXT: .cfi_offset 31, -4
+; MM32R3-NEXT: .cfi_offset 30, -8
+; MM32R3-NEXT: move $fp, $sp
+; MM32R3-NEXT: .cfi_def_cfa_register 30
+; MM32R3-NEXT: addiu $1, $zero, -16
+; MM32R3-NEXT: and $sp, $sp, $1
+; MM32R3-NEXT: lwp $2, 32($fp)
; MM32R3-NEXT: xor16 $2, $4
; MM32R3-NEXT: xor16 $3, $5
-; MM32R3-NEXT: lw $4, 24($sp)
+; MM32R3-NEXT: lw $4, 40($fp)
; MM32R3-NEXT: xor16 $4, $6
-; MM32R3-NEXT: lw $5, 28($sp)
+; MM32R3-NEXT: lw $5, 44($fp)
; MM32R3-NEXT: xor16 $5, $7
+; MM32R3-NEXT: move $sp, $fp
+; MM32R3-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MM32R3-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MM32R3-NEXT: addiusp 16
; MM32R3-NEXT: jrc $ra
;
; MM32R6-LABEL: xor_i128:
; MM32R6: # %bb.0: # %entry
-; MM32R6-NEXT: lw $1, 20($sp)
-; MM32R6-NEXT: lw $2, 16($sp)
+; MM32R6-NEXT: addiu $sp, $sp, -16
+; MM32R6-NEXT: .cfi_def_cfa_offset 16
+; MM32R6-NEXT: sw $ra, 12($sp) # 4-byte Folded Spill
+; MM32R6-NEXT: sw $fp, 8($sp) # 4-byte Folded Spill
+; MM32R6-NEXT: .cfi_offset 31, -4
+; MM32R6-NEXT: .cfi_offset 30, -8
+; MM32R6-NEXT: move $fp, $sp
+; MM32R6-NEXT: .cfi_def_cfa_register 30
+; MM32R6-NEXT: addiu $1, $zero, -16
+; MM32R6-NEXT: and $sp, $sp, $1
+; MM32R6-NEXT: lw $1, 36($fp)
+; MM32R6-NEXT: lw $2, 32($fp)
; MM32R6-NEXT: xor $2, $4, $2
; MM32R6-NEXT: xor $3, $5, $1
-; MM32R6-NEXT: lw $1, 24($sp)
+; MM32R6-NEXT: lw $1, 40($fp)
; MM32R6-NEXT: xor $4, $6, $1
-; MM32R6-NEXT: lw $1, 28($sp)
+; MM32R6-NEXT: lw $1, 44($fp)
; MM32R6-NEXT: xor $5, $7, $1
+; MM32R6-NEXT: move $sp, $fp
+; MM32R6-NEXT: lw $fp, 8($sp) # 4-byte Folded Reload
+; MM32R6-NEXT: lw $ra, 12($sp) # 4-byte Folded Reload
+; MM32R6-NEXT: addiu $sp, $sp, 16
; MM32R6-NEXT: jrc $ra
entry:
%r = xor i128 %a, %b
diff --git a/llvm/test/CodeGen/Mips/llvm.sincos.ll b/llvm/test/CodeGen/Mips/llvm.sincos.ll
index 58eb427d3a69b..204a447955385 100644
--- a/llvm/test/CodeGen/Mips/llvm.sincos.ll
+++ b/llvm/test/CodeGen/Mips/llvm.sincos.ll
@@ -768,77 +768,89 @@ define { <2 x double>, <2 x double> } @test_sincos_v2f64(<2 x double> %a) #0 {
define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
; MIPSEL-LABEL: test_sincos_f128:
; MIPSEL: # %bb.0:
-; MIPSEL-NEXT: addiu $sp, $sp, -64
-; MIPSEL-NEXT: sw $ra, 60($sp) # 4-byte Folded Spill
-; MIPSEL-NEXT: sw $16, 56($sp) # 4-byte Folded Spill
+; MIPSEL-NEXT: addiu $sp, $sp, -80
+; MIPSEL-NEXT: sw $ra, 76($sp) # 4-byte Folded Spill
+; MIPSEL-NEXT: sw $fp, 72($sp) # 4-byte Folded Spill
+; MIPSEL-NEXT: sw $16, 68($sp) # 4-byte Folded Spill
+; MIPSEL-NEXT: move $fp, $sp
+; MIPSEL-NEXT: addiu $1, $zero, -16
+; MIPSEL-NEXT: and $sp, $sp, $1
; MIPSEL-NEXT: move $1, $7
; MIPSEL-NEXT: move $16, $4
-; MIPSEL-NEXT: addiu $2, $sp, 24
+; MIPSEL-NEXT: addiu $2, $sp, 32
; MIPSEL-NEXT: sw $2, 20($sp)
-; MIPSEL-NEXT: addiu $2, $sp, 40
+; MIPSEL-NEXT: addiu $2, $sp, 48
; MIPSEL-NEXT: sw $2, 16($sp)
-; MIPSEL-NEXT: lw $7, 80($sp)
+; MIPSEL-NEXT: lw $7, 96($fp)
; MIPSEL-NEXT: move $4, $5
; MIPSEL-NEXT: move $5, $6
; MIPSEL-NEXT: jal sincosl
; MIPSEL-NEXT: move $6, $1
-; MIPSEL-NEXT: lw $1, 52($sp)
-; MIPSEL-NEXT: lw $2, 24($sp)
-; MIPSEL-NEXT: lw $3, 28($sp)
-; MIPSEL-NEXT: lw $4, 32($sp)
-; MIPSEL-NEXT: lw $5, 36($sp)
+; MIPSEL-NEXT: lw $1, 60($sp)
+; MIPSEL-NEXT: lw $2, 32($sp)
+; MIPSEL-NEXT: lw $3, 36($sp)
+; MIPSEL-NEXT: lw $4, 40($sp)
+; MIPSEL-NEXT: lw $5, 44($sp)
; MIPSEL-NEXT: sw $5, 28($16)
; MIPSEL-NEXT: sw $4, 24($16)
; MIPSEL-NEXT: sw $3, 20($16)
; MIPSEL-NEXT: sw $2, 16($16)
; MIPSEL-NEXT: sw $1, 12($16)
-; MIPSEL-NEXT: lw $1, 48($sp)
+; MIPSEL-NEXT: lw $1, 56($sp)
; MIPSEL-NEXT: sw $1, 8($16)
-; MIPSEL-NEXT: lw $1, 44($sp)
+; MIPSEL-NEXT: lw $1, 52($sp)
; MIPSEL-NEXT: sw $1, 4($16)
-; MIPSEL-NEXT: lw $1, 40($sp)
+; MIPSEL-NEXT: lw $1, 48($sp)
; MIPSEL-NEXT: sw $1, 0($16)
-; MIPSEL-NEXT: lw $16, 56($sp) # 4-byte Folded Reload
-; MIPSEL-NEXT: lw $ra, 60($sp) # 4-byte Folded Reload
+; MIPSEL-NEXT: move $sp, $fp
+; MIPSEL-NEXT: lw $16, 68($sp) # 4-byte Folded Reload
+; MIPSEL-NEXT: lw $fp, 72($sp) # 4-byte Folded Reload
+; MIPSEL-NEXT: lw $ra, 76($sp) # 4-byte Folded Reload
; MIPSEL-NEXT: jr $ra
-; MIPSEL-NEXT: addiu $sp, $sp, 64
+; MIPSEL-NEXT: addiu $sp, $sp, 80
;
; SOFT-FLOAT-32-LABEL: test_sincos_f128:
; SOFT-FLOAT-32: # %bb.0:
-; SOFT-FLOAT-32-NEXT: addiu $sp, $sp, -64
-; SOFT-FLOAT-32-NEXT: sw $ra, 60($sp) # 4-byte Folded Spill
-; SOFT-FLOAT-32-NEXT: sw $16, 56($sp) # 4-byte Folded Spill
+; SOFT-FLOAT-32-NEXT: addiu $sp, $sp, -80
+; SOFT-FLOAT-32-NEXT: sw $ra, 76($sp) # 4-byte Folded Spill
+; SOFT-FLOAT-32-NEXT: sw $fp, 72($sp) # 4-byte Folded Spill
+; SOFT-FLOAT-32-NEXT: sw $16, 68($sp) # 4-byte Folded Spill
+; SOFT-FLOAT-32-NEXT: move $fp, $sp
+; SOFT-FLOAT-32-NEXT: addiu $1, $zero, -16
+; SOFT-FLOAT-32-NEXT: and $sp, $sp, $1
; SOFT-FLOAT-32-NEXT: move $1, $7
; SOFT-FLOAT-32-NEXT: move $16, $4
-; SOFT-FLOAT-32-NEXT: addiu $2, $sp, 24
+; SOFT-FLOAT-32-NEXT: addiu $2, $sp, 32
; SOFT-FLOAT-32-NEXT: sw $2, 20($sp)
-; SOFT-FLOAT-32-NEXT: addiu $2, $sp, 40
+; SOFT-FLOAT-32-NEXT: addiu $2, $sp, 48
; SOFT-FLOAT-32-NEXT: sw $2, 16($sp)
-; SOFT-FLOAT-32-NEXT: lw $7, 80($sp)
+; SOFT-FLOAT-32-NEXT: lw $7, 96($fp)
; SOFT-FLOAT-32-NEXT: move $4, $5
; SOFT-FLOAT-32-NEXT: move $5, $6
; SOFT-FLOAT-32-NEXT: jal sincosl
; SOFT-FLOAT-32-NEXT: move $6, $1
-; SOFT-FLOAT-32-NEXT: lw $1, 52($sp)
-; SOFT-FLOAT-32-NEXT: lw $2, 24($sp)
-; SOFT-FLOAT-32-NEXT: lw $3, 28($sp)
-; SOFT-FLOAT-32-NEXT: lw $4, 32($sp)
-; SOFT-FLOAT-32-NEXT: lw $5, 36($sp)
+; SOFT-FLOAT-32-NEXT: lw $1, 60($sp)
+; SOFT-FLOAT-32-NEXT: lw $2, 32($sp)
+; SOFT-FLOAT-32-NEXT: lw $3, 36($sp)
+; SOFT-FLOAT-32-NEXT: lw $4, 40($sp)
+; SOFT-FLOAT-32-NEXT: lw $5, 44($sp)
; SOFT-FLOAT-32-NEXT: sw $5, 28($16)
; SOFT-FLOAT-32-NEXT: sw $4, 24($16)
; SOFT-FLOAT-32-NEXT: sw $3, 20($16)
; SOFT-FLOAT-32-NEXT: sw $2, 16($16)
; SOFT-FLOAT-32-NEXT: sw $1, 12($16)
-; SOFT-FLOAT-32-NEXT: lw $1, 48($sp)
+; SOFT-FLOAT-32-NEXT: lw $1, 56($sp)
; SOFT-FLOAT-32-NEXT: sw $1, 8($16)
-; SOFT-FLOAT-32-NEXT: lw $1, 44($sp)
+; SOFT-FLOAT-32-NEXT: lw $1, 52($sp)
; SOFT-FLOAT-32-NEXT: sw $1, 4($16)
-; SOFT-FLOAT-32-NEXT: lw $1, 40($sp)
+; SOFT-FLOAT-32-NEXT: lw $1, 48($sp)
; SOFT-FLOAT-32-NEXT: sw $1, 0($16)
-; SOFT-FLOAT-32-NEXT: lw $16, 56($sp) # 4-byte Folded Reload
-; SOFT-FLOAT-32-NEXT: lw $ra, 60($sp) # 4-byte Folded Reload
+; SOFT-FLOAT-32-NEXT: move $sp, $fp
+; SOFT-FLOAT-32-NEXT: lw $16, 68($sp) # 4-byte Folded Reload
+; SOFT-FLOAT-32-NEXT: lw $fp, 72($sp) # 4-byte Folded Reload
+; SOFT-FLOAT-32-NEXT: lw $ra, 76($sp) # 4-byte Folded Reload
; SOFT-FLOAT-32-NEXT: jr $ra
-; SOFT-FLOAT-32-NEXT: addiu $sp, $sp, 64
+; SOFT-FLOAT-32-NEXT: addiu $sp, $sp, 80
;
; SOFT-FLOAT-64-LABEL: test_sincos_f128:
; SOFT-FLOAT-64: # %bb.0:
@@ -861,43 +873,47 @@ define { fp128, fp128 } @test_sincos_f128(fp128 %a) #0 {
define { <2 x fp128>, <2 x fp128> } @test_sincos_v2f128(<2 x fp128> %a) #0 {
; MIPSEL-LABEL: test_sincos_v2f128:
; MIPSEL: # %bb.0:
-; MIPSEL-NEXT: addiu $sp, $sp, -96
-; MIPSEL-NEXT: sw $ra, 92($sp) # 4-byte Folded Spill
-; MIPSEL-NEXT: sw $16, 88($sp) # 4-byte Folded Spill
+; MIPSEL-NEXT: addiu $sp, $sp, -112
+; MIPSEL-NEXT: sw $ra, 108($sp) # 4-byte Folded Spill
+; MIPSEL-NEXT: sw $fp, 104($sp) # 4-byte Folded Spill
+; MIPSEL-NEXT: sw $16, 100($sp) # 4-byte Folded Spill
+; MIPSEL-NEXT: move $fp, $sp
+; MIPSEL-NEXT: addiu $1, $zero, -16
+; MIPSEL-NEXT: and $sp, $sp, $1
; MIPSEL-NEXT: move $5, $7
; MIPSEL-NEXT: move $1, $6
; MIPSEL-NEXT: move $16, $4
-; MIPSEL-NEXT: addiu $2, $sp, 24
+; MIPSEL-NEXT: addiu $2, $sp, 32
; MIPSEL-NEXT: sw $2, 20($sp)
-; MIPSEL-NEXT: addiu $2, $sp, 40
+; MIPSEL-NEXT: addiu $2, $sp, 48
; MIPSEL-NEXT: sw $2, 16($sp)
-; MIPSEL-NEXT: lw $6, 112($sp)
-; MIPSEL-NEXT: lw $7, 116($sp)
+; MIPSEL-NEXT: lw $6, 128($fp)
+; MIPSEL-NEXT: lw $7, 132($fp)
; MIPSEL-NEXT: jal sincosl
; MIPSEL-NEXT: move $4, $1
-; MIPSEL-NEXT: addiu $1, $sp, 56
+; MIPSEL-NEXT: addiu $1, $sp, 64
; MIPSEL-NEXT: sw $1, 20($sp)
-; MIPSEL-NEXT: addiu $1, $sp, 72
+; MIPSEL-NEXT: addiu $1, $sp, 80
; MIPSEL-NEXT: sw $1, 16($sp)
-; MIPSEL-NEXT: lw $4, 120($sp)
-; MIPSEL-NEXT: lw $5, 124($sp)
-; MIPSEL-NEXT: lw $6, 128($sp)
-; MIPSEL-NEXT: lw $7, 132($sp)
+; MIPSEL-NEXT: lw $4, 136($fp)
+; MIPSEL-NEXT: lw $5, 140($fp)
+; MIPSEL-NEXT: lw $6, 144($fp)
+; MIPSEL-NEXT: lw $7, 148($fp)
; MIPSEL-NEXT: jal sincosl
; MIPSEL-NEXT: nop
-; MIPSEL-NEXT: lw $1, 36($sp)
-; MIPSEL-NEXT: lw $2, 56($sp)
-; MIPSEL-NEXT: lw $3, 60($sp)
-; MIPSEL-NEXT: lw $4, 64($sp)
-; MIPSEL-NEXT: lw $5, 52($sp)
-; MIPSEL-NEXT: lw $6, 72($sp)
-; MIPSEL-NEXT: lw $7, 76($sp)
-; MIPSEL-NEXT: lw $8, 80($sp)
-; MIPSEL-NEXT: lw $9, 84($sp)
-; MIPSEL-NEXT: lw $10, 24($sp)
-; MIPSEL-NEXT: lw $11, 28($sp)
-; MIPSEL-NEXT: lw $12, 32($sp)
-; MIPSEL-NEXT: lw $13, 68($sp)
+; MIPSEL-NEXT: lw $1, 44($sp)
+; MIPSEL-NEXT: lw $2, 64($sp)
+; MIPSEL-NEXT: lw $3, 68($sp)
+; MIPSEL-NEXT: lw $4, 72($sp)
+; MIPSEL-NEXT: lw $5, 60($sp)
+; MIPSEL-NEXT: lw $6, 80($sp)
+; MIPSEL-NEXT: lw $7, 84($sp)
+; MIPSEL-NEXT: lw $8, 88($sp)
+; MIPSEL-NEXT: lw $9, 92($sp)
+; MIPSEL-NEXT: lw $10, 32($sp)
+; MIPSEL-NEXT: lw $11, 36($sp)
+; MIPSEL-NEXT: lw $12, 40($sp)
+; MIPSEL-NEXT: lw $13, 76($sp)
; MIPSEL-NEXT: sw $13, 60($16)
; MIPSEL-NEXT: sw $4, 56($16)
; MIPSEL-NEXT: sw $3, 52($16)
@@ -911,56 +927,62 @@ define { <2 x fp128>, <2 x fp128> } @test_sincos_v2f128(<2 x fp128> %a) #0 {
; MIPSEL-NEXT: sw $7, 20($16)
; MIPSEL-NEXT: sw $6, 16($16)
; MIPSEL-NEXT: sw $5, 12($16)
-; MIPSEL-NEXT: lw $1, 48($sp)
+; MIPSEL-NEXT: lw $1, 56($sp)
; MIPSEL-NEXT: sw $1, 8($16)
-; MIPSEL-NEXT: lw $1, 44($sp)
+; MIPSEL-NEXT: lw $1, 52($sp)
; MIPSEL-NEXT: sw $1, 4($16)
-; MIPSEL-NEXT: lw $1, 40($sp)
+; MIPSEL-NEXT: lw $1, 48($sp)
; MIPSEL-NEXT: sw $1, 0($16)
-; MIPSEL-NEXT: lw $16, 88($sp) # 4-byte Folded Reload
-; MIPSEL-NEXT: lw $ra, 92($sp) # 4-byte Folded Reload
+; MIPSEL-NEXT: move $sp, $fp
+; MIPSEL-NEXT: lw $16, 100($sp) # 4-byte Folded Reload
+; MIPSEL-NEXT: lw $fp, 104($sp) # 4-byte Folded Reload
+; MIPSEL-NEXT: lw $ra, 108($sp) # 4-byte Folded Reload
; MIPSEL-NEXT: jr $ra
-; MIPSEL-NEXT: addiu $sp, $sp, 96
+; MIPSEL-NEXT: addiu $sp, $sp, 112
;
; SOFT-FLOAT-32-LABEL: test_sincos_v2f128:
; SOFT-FLOAT-32: # %bb.0:
-; SOFT-FLOAT-32-NEXT: addiu $sp, $sp, -96
-; SOFT-FLOAT-32-NEXT: sw $ra, 92($sp) # 4-byte Folded Spill
-; SOFT-FLOAT-32-NEXT: sw $16, 88($sp) # 4-byte Folded Spill
+; SOFT-FLOAT-32-NEXT: addiu $sp, $sp, -112
+; SOFT-FLOAT-32-NEXT: sw $ra, 108($sp) # 4-byte Folded Spill
+; SOFT-FLOAT-32-NEXT: sw $fp, 104($sp) # 4-byte Folded Spill
+; SOFT-FLOAT-32-NEXT: sw $16, 100($sp) # 4-byte Folded Spill
+; SOFT-FLOAT-32-NEXT: move $fp, $sp
+; SOFT-FLOAT-32-NEXT: addiu $1, $zero, -16
+; SOFT-FLOAT-32-NEXT: and $sp, $sp, $1
; SOFT-FLOAT-32-NEXT: move $5, $7
; SOFT-FLOAT-32-NEXT: move $1, $6
; SOFT-FLOAT-32-NEXT: move $16, $4
-; SOFT-FLOAT-32-NEXT: addiu $2, $sp, 24
+; SOFT-FLOAT-32-NEXT: addiu $2, $sp, 32
; SOFT-FLOAT-32-NEXT: sw $2, 20($sp)
-; SOFT-FLOAT-32-NEXT: addiu $2, $sp, 40
+; SOFT-FLOAT-32-NEXT: addiu $2, $sp, 48
; SOFT-FLOAT-32-NEXT: sw $2, 16($sp)
-; SOFT-FLOAT-32-NEXT: lw $6, 112($sp)
-; SOFT-FLOAT-32-NEXT: lw $7, 116($sp)
+; SOFT-FLOAT-32-NEXT: lw $6, 128($fp)
+; SOFT-FLOAT-32-NEXT: lw $7, 132($fp)
; SOFT-FLOAT-32-NEXT: jal sincosl
; SOFT-FLOAT-32-NEXT: move $4, $1
-; SOFT-FLOAT-32-NEXT: addiu $1, $sp, 56
+; SOFT-FLOAT-32-NEXT: addiu $1, $sp, 64
; SOFT-FLOAT-32-NEXT: sw $1, 20($sp)
-; SOFT-FLOAT-32-NEXT: addiu $1, $sp, 72
+; SOFT-FLOAT-32-NEXT: addiu $1, $sp, 80
; SOFT-FLOAT-32-NEXT: sw $1, 16($sp)
-; SOFT-FLOAT-32-NEXT: lw $4, 120($sp)
-; SOFT-FLOAT-32-NEXT: lw $5, 124($sp)
-; SOFT-FLOAT-32-NEXT: lw $6, 128($sp)
-; SOFT-FLOAT-32-NEXT: lw $7, 132($sp)
+; SOFT-FLOAT-32-NEXT: lw $4, 136($fp)
+; SOFT-FLOAT-32-NEXT: lw $5, 140($fp)
+; SOFT-FLOAT-32-NEXT: lw $6, 144($fp)
+; SOFT-FLOAT-32-NEXT: lw $7, 148($fp)
; SOFT-FLOAT-32-NEXT: jal sincosl
; SOFT-FLOAT-32-NEXT: nop
-; SOFT-FLOAT-32-NEXT: lw $1, 36($sp)
-; SOFT-FLOAT-32-NEXT: lw $2, 56($sp)
-; SOFT-FLOAT-32-NEXT: lw $3, 60($sp)
-; SOFT-FLOAT-32-NEXT: lw $4, 64($sp)
-; SOFT-FLOAT-32-NEXT: lw $5, 52($sp)
-; SOFT-FLOAT-32-NEXT: lw $6, 72($sp)
-; SOFT-FLOAT-32-NEXT: lw $7, 76($sp)
-; SOFT-FLOAT-32-NEXT: lw $8, 80($sp)
-; SOFT-FLOAT-32-NEXT: lw $9, 84($sp)
-; SOFT-FLOAT-32-NEXT: lw $10, 24($sp)
-; SOFT-FLOAT-32-NEXT: lw $11, 28($sp)
-; SOFT-FLOAT-32-NEXT: lw $12, 32($sp)
-; SOFT-FLOAT-32-NEXT: lw $13, 68($sp)
+; SOFT-FLOAT-32-NEXT: lw $1, 44($sp)
+; SOFT-FLOAT-32-NEXT: lw $2, 64($sp)
+; SOFT-FLOAT-32-NEXT: lw $3, 68($sp)
+; SOFT-FLOAT-32-NEXT: lw $4, 72($sp)
+; SOFT-FLOAT-32-NEXT: lw $5, 60($sp)
+; SOFT-FLOAT-32-NEXT: lw $6, 80($sp)
+; SOFT-FLOAT-32-NEXT: lw $7, 84($sp)
+; SOFT-FLOAT-32-NEXT: lw $8, 88($sp)
+; SOFT-FLOAT-32-NEXT: lw $9, 92($sp)
+; SOFT-FLOAT-32-NEXT: lw $10, 32($sp)
+; SOFT-FLOAT-32-NEXT: lw $11, 36($sp)
+; SOFT-FLOAT-32-NEXT: lw $12, 40($sp)
+; SOFT-FLOAT-32-NEXT: lw $13, 76($sp)
; SOFT-FLOAT-32-NEXT: sw $13, 60($16)
; SOFT-FLOAT-32-NEXT: sw $4, 56($16)
; SOFT-FLOAT-32-NEXT: sw $3, 52($16)
@@ -974,16 +996,18 @@ define { <2 x fp128>, <2 x fp128> } @test_sincos_v2f128(<2 x fp128> %a) #0 {
; SOFT-FLOAT-32-NEXT: sw $7, 20($16)
; SOFT-FLOAT-32-NEXT: sw $6, 16($16)
; SOFT-FLOAT-32-NEXT: sw $5, 12($16)
-; SOFT-FLOAT-32-NEXT: lw $1, 48($sp)
+; SOFT-FLOAT-32-NEXT: lw $1, 56($sp)
; SOFT-FLOAT-32-NEXT: sw $1, 8($16)
-; SOFT-FLOAT-32-NEXT: lw $1, 44($sp)
+; SOFT-FLOAT-32-NEXT: lw $1, 52($sp)
; SOFT-FLOAT-32-NEXT: sw $1, 4($16)
-; SOFT-FLOAT-32-NEXT: lw $1, 40($sp)
+; SOFT-FLOAT-32-NEXT: lw $1, 48($sp)
; SOFT-FLOAT-32-NEXT: sw $1, 0($16)
-; SOFT-FLOAT-32-NEXT: lw $16, 88($sp) # 4-byte Folded Reload
-; SOFT-FLOAT-32-NEXT: lw $ra, 92($sp) # 4-byte Folded Reload
+; SOFT-FLOAT-32-NEXT: move $sp, $fp
+; SOFT-FLOAT-32-NEXT: lw $16, 100($sp) # 4-byte Folded Reload
+; SOFT-FLOAT-32-NEXT: lw $fp, 104($sp) # 4-byte Folded Reload
+; SOFT-FLOAT-32-NEXT: lw $ra, 108($sp) # 4-byte Folded Reload
; SOFT-FLOAT-32-NEXT: jr $ra
-; SOFT-FLOAT-32-NEXT: addiu $sp, $sp, 96
+; SOFT-FLOAT-32-NEXT: addiu $sp, $sp, 112
;
; SOFT-FLOAT-64-LABEL: test_sincos_v2f128:
; SOFT-FLOAT-64: # %bb.0:
diff --git a/llvm/test/CodeGen/PowerPC/all-atomics.ll b/llvm/test/CodeGen/PowerPC/all-atomics.ll
index 17b3b75a66800..c80a59d659cb2 100644
--- a/llvm/test/CodeGen/PowerPC/all-atomics.ll
+++ b/llvm/test/CodeGen/PowerPC/all-atomics.ll
@@ -509,31 +509,31 @@ define dso_local void @test_op_ignore() local_unnamed_addr #0 {
; AIX32-LABEL: test_op_ignore:
; AIX32: # %bb.0: # %entry
; AIX32-NEXT: mflr 0
-; AIX32-NEXT: stwu 1, -160(1)
+; AIX32-NEXT: stwu 1, -176(1)
; AIX32-NEXT: lwz 3, L..C0(2) # @sc
-; AIX32-NEXT: stw 0, 168(1)
+; AIX32-NEXT: stw 0, 184(1)
; AIX32-NEXT: rlwinm 4, 3, 3, 27, 28
-; AIX32-NEXT: stw 15, 92(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 26, 136(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 28, 144(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 15, 108(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 26, 152(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 28, 160(1) # 4-byte Folded Spill
; AIX32-NEXT: li 15, 1
; AIX32-NEXT: rlwinm 28, 3, 0, 0, 29
; AIX32-NEXT: li 3, 255
; AIX32-NEXT: xori 26, 4, 24
-; AIX32-NEXT: stw 16, 96(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 17, 100(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 18, 104(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 19, 108(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 20, 112(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 21, 116(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 22, 120(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 23, 124(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 24, 128(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 25, 132(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 27, 140(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 29, 148(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 30, 152(1) # 4-byte Folded Spill
-; AIX32-NEXT: stw 31, 156(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 16, 112(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 17, 116(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 18, 120(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 19, 124(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 20, 128(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 21, 132(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 22, 136(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 23, 140(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 24, 144(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 25, 148(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 27, 156(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 29, 164(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 30, 168(1) # 4-byte Folded Spill
+; AIX32-NEXT: stw 31, 172(1) # 4-byte Folded Spill
; AIX32-NEXT: sync
; AIX32-NEXT: slw 29, 15, 26
; AIX32-NEXT: slw 3, 3, 26
@@ -906,82 +906,82 @@ define dso_local void @test_op_ignore() local_unnamed_addr #0 {
; AIX32-NEXT: bl .__atomic_fetch_xor_8[PR]
; AIX32-NEXT: nop
; AIX32-NEXT: lwz 31, L..C8(2) # @u128
-; AIX32-NEXT: addi 30, 1, 72
+; AIX32-NEXT: addi 30, 1, 80
; AIX32-NEXT: li 3, 16
; AIX32-NEXT: mr 5, 30
; AIX32-NEXT: li 6, 0
; AIX32-NEXT: mr 4, 31
; AIX32-NEXT: bl .__atomic_load[PR]
; AIX32-NEXT: nop
-; AIX32-NEXT: lwz 5, 84(1)
-; AIX32-NEXT: lwz 4, 80(1)
-; AIX32-NEXT: lwz 6, 76(1)
-; AIX32-NEXT: lwz 7, 72(1)
-; AIX32-NEXT: addi 29, 1, 56
+; AIX32-NEXT: lwz 5, 92(1)
+; AIX32-NEXT: lwz 4, 88(1)
+; AIX32-NEXT: lwz 6, 84(1)
+; AIX32-NEXT: lwz 7, 80(1)
+; AIX32-NEXT: addi 29, 1, 64
; AIX32-NEXT: .align 4
; AIX32-NEXT: L..BB0_49: # %atomicrmw.start2
; AIX32-NEXT: #
; AIX32-NEXT: xori 3, 5, 1
-; AIX32-NEXT: stw 7, 72(1)
-; AIX32-NEXT: stw 7, 56(1)
+; AIX32-NEXT: stw 7, 80(1)
+; AIX32-NEXT: stw 7, 64(1)
; AIX32-NEXT: li 7, 5
-; AIX32-NEXT: stw 3, 68(1)
+; AIX32-NEXT: stw 3, 76(1)
; AIX32-NEXT: li 3, 16
; AIX32-NEXT: li 8, 5
-; AIX32-NEXT: stw 6, 76(1)
-; AIX32-NEXT: stw 4, 80(1)
-; AIX32-NEXT: stw 5, 84(1)
-; AIX32-NEXT: stw 4, 64(1)
-; AIX32-NEXT: stw 6, 60(1)
+; AIX32-NEXT: stw 6, 84(1)
+; AIX32-NEXT: stw 4, 88(1)
+; AIX32-NEXT: stw 5, 92(1)
+; AIX32-NEXT: stw 4, 72(1)
+; AIX32-NEXT: stw 6, 68(1)
; AIX32-NEXT: mr 4, 31
; AIX32-NEXT: mr 5, 30
; AIX32-NEXT: mr 6, 29
; AIX32-NEXT: bl .__atomic_compare_exchange[PR]
; AIX32-NEXT: nop
-; AIX32-NEXT: lwz 5, 84(1)
-; AIX32-NEXT: lwz 4, 80(1)
-; AIX32-NEXT: lwz 6, 76(1)
-; AIX32-NEXT: lwz 7, 72(1)
+; AIX32-NEXT: lwz 5, 92(1)
+; AIX32-NEXT: lwz 4, 88(1)
+; AIX32-NEXT: lwz 6, 84(1)
+; AIX32-NEXT: lwz 7, 80(1)
; AIX32-NEXT: cmplwi 3, 0
; AIX32-NEXT: beq 0, L..BB0_49
; AIX32-NEXT: # %bb.50: # %atomicrmw.end1
; AIX32-NEXT: lwz 31, L..C9(2) # @s128
-; AIX32-NEXT: addi 30, 1, 72
+; AIX32-NEXT: addi 30, 1, 80
; AIX32-NEXT: li 3, 16
; AIX32-NEXT: mr 5, 30
; AIX32-NEXT: li 6, 0
; AIX32-NEXT: mr 4, 31
; AIX32-NEXT: bl .__atomic_load[PR]
; AIX32-NEXT: nop
-; AIX32-NEXT: lwz 5, 84(1)
-; AIX32-NEXT: lwz 4, 80(1)
-; AIX32-NEXT: lwz 6, 76(1)
-; AIX32-NEXT: lwz 7, 72(1)
-; AIX32-NEXT: addi 29, 1, 56
+; AIX32-NEXT: lwz 5, 92(1)
+; AIX32-NEXT: lwz 4, 88(1)
+; AIX32-NEXT: lwz 6, 84(1)
+; AIX32-NEXT: lwz 7, 80(1)
+; AIX32-NEXT: addi 29, 1, 64
; AIX32-NEXT: .align 4
; AIX32-NEXT: L..BB0_51: # %atomicrmw.start
; AIX32-NEXT: #
; AIX32-NEXT: xori 3, 5, 1
-; AIX32-NEXT: stw 7, 72(1)
-; AIX32-NEXT: stw 7, 56(1)
+; AIX32-NEXT: stw 7, 80(1)
+; AIX32-NEXT: stw 7, 64(1)
; AIX32-NEXT: li 7, 5
-; AIX32-NEXT: stw 3, 68(1)
+; AIX32-NEXT: stw 3, 76(1)
; AIX32-NEXT: li 3, 16
; AIX32-NEXT: li 8, 5
-; AIX32-NEXT: stw 6, 76(1)
-; AIX32-NEXT: stw 4, 80(1)
-; AIX32-NEXT: stw 5, 84(1)
-; AIX32-NEXT: stw 4, 64(1)
-; AIX32-NEXT: stw 6, 60(1)
+; AIX32-NEXT: stw 6, 84(1)
+; AIX32-NEXT: stw 4, 88(1)
+; AIX32-NEXT: stw 5, 92(1)
+; AIX32-NEXT: stw 4, 72(1)
+; AIX32-NEXT: stw 6, 68(1)
; AIX32-NEXT: mr 4, 31
; AIX32-NEXT: mr 5, 30
; AIX32-NEXT: mr 6, 29
; AIX32-NEXT: bl .__atomic_compare_exchange[PR]
; AIX32-NEXT: nop
-; AIX32-NEXT: lwz 5, 84(1)
-; AIX32-NEXT: lwz 4, 80(1)
-; AIX32-NEXT: lwz 6, 76(1)
-; AIX32-NEXT: lwz 7, 72(1)
+; AIX32-NEXT: lwz 5, 92(1)
+; AIX32-NEXT: lwz 4, 88(1)
+; AIX32-NEXT: lwz 6, 84(1)
+; AIX32-NEXT: lwz 7, 80(1)
; AIX32-NEXT: cmplwi 3, 0
; AIX32-NEXT: beq 0, L..BB0_51
; AIX32-NEXT: # %bb.52: # %atomicrmw.end
@@ -1168,24 +1168,24 @@ define dso_local void @test_op_ignore() local_unnamed_addr #0 {
; AIX32-NEXT: li 6, 5
; AIX32-NEXT: bl .__atomic_fetch_and_8[PR]
; AIX32-NEXT: nop
-; AIX32-NEXT: lwz 31, 156(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 30, 152(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 29, 148(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 28, 144(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 27, 140(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 26, 136(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 25, 132(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 24, 128(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 23, 124(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 22, 120(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 21, 116(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 20, 112(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 19, 108(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 18, 104(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 17, 100(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 16, 96(1) # 4-byte Folded Reload
-; AIX32-NEXT: lwz 15, 92(1) # 4-byte Folded Reload
-; AIX32-NEXT: addi 1, 1, 160
+; AIX32-NEXT: lwz 31, 172(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 30, 168(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 29, 164(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 28, 160(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 27, 156(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 26, 152(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 25, 148(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 24, 144(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 23, 140(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 22, 136(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 21, 132(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 20, 128(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 19, 124(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 18, 120(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 17, 116(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 16, 112(1) # 4-byte Folded Reload
+; AIX32-NEXT: lwz 15, 108(1) # 4-byte Folded Reload
+; AIX32-NEXT: addi 1, 1, 176
; AIX32-NEXT: lwz 0, 8(1)
; AIX32-NEXT: mtlr 0
; AIX32-NEXT: blr
diff --git a/llvm/test/CodeGen/PowerPC/ctrloop-sh.ll b/llvm/test/CodeGen/PowerPC/ctrloop-sh.ll
index 72de456cba395..19f86f9d1af6f 100644
--- a/llvm/test/CodeGen/PowerPC/ctrloop-sh.ll
+++ b/llvm/test/CodeGen/PowerPC/ctrloop-sh.ll
@@ -16,39 +16,41 @@ define void @foo1(ptr %a, ptr readonly %b, ptr readonly %c) #0 {
; CHECK-NEXT: addi 7, 1, 16
; CHECK-NEXT: .LBB0_1: # %for.body
; CHECK-NEXT: #
-; CHECK-NEXT: lwz 8, 0(4)
; CHECK-NEXT: lwz 9, 4(4)
+; CHECK-NEXT: lwz 8, 0(4)
; CHECK-NEXT: lwz 10, 8(4)
; CHECK-NEXT: lwz 11, 12(4)
; CHECK-NEXT: lwz 12, 12(5)
+; CHECK-NEXT: stw 9, 20(1)
+; CHECK-NEXT: mr 9, 7
; CHECK-NEXT: stw 6, 44(1)
+; CHECK-NEXT: rlwimi 9, 12, 29, 28, 29
; CHECK-NEXT: stw 6, 40(1)
; CHECK-NEXT: stw 6, 36(1)
; CHECK-NEXT: stw 6, 32(1)
; CHECK-NEXT: stw 11, 28(1)
; CHECK-NEXT: stw 10, 24(1)
; CHECK-NEXT: clrlwi 10, 12, 27
-; CHECK-NEXT: stw 9, 20(1)
; CHECK-NEXT: stw 8, 16(1)
-; CHECK-NEXT: rlwinm 8, 12, 29, 28, 29
-; CHECK-NEXT: lwzux 9, 8, 7
-; CHECK-NEXT: subfic 12, 10, 32
-; CHECK-NEXT: lwz 11, 8(8)
-; CHECK-NEXT: slw 9, 9, 10
-; CHECK-NEXT: lwz 0, 4(8)
-; CHECK-NEXT: lwz 8, 12(8)
-; CHECK-NEXT: srw 30, 11, 12
-; CHECK-NEXT: slw 29, 0, 10
-; CHECK-NEXT: srw 0, 0, 12
-; CHECK-NEXT: srw 12, 8, 12
-; CHECK-NEXT: slw 11, 11, 10
+; CHECK-NEXT: rlwinm 12, 12, 29, 28, 29
+; CHECK-NEXT: lwz 8, 8(9)
+; CHECK-NEXT: subfic 0, 10, 32
+; CHECK-NEXT: lwz 11, 4(9)
+; CHECK-NEXT: lwz 9, 12(9)
+; CHECK-NEXT: srw 30, 8, 0
+; CHECK-NEXT: lwzx 12, 7, 12
+; CHECK-NEXT: slw 29, 11, 10
+; CHECK-NEXT: srw 11, 11, 0
+; CHECK-NEXT: srw 0, 9, 0
; CHECK-NEXT: slw 8, 8, 10
-; CHECK-NEXT: stw 8, 12(3)
-; CHECK-NEXT: or 8, 11, 12
+; CHECK-NEXT: slw 12, 12, 10
+; CHECK-NEXT: or 8, 8, 0
; CHECK-NEXT: stw 8, 8(3)
-; CHECK-NEXT: or 8, 9, 0
+; CHECK-NEXT: or 8, 12, 11
+; CHECK-NEXT: slw 9, 9, 10
; CHECK-NEXT: stw 8, 0(3)
; CHECK-NEXT: or 8, 29, 30
+; CHECK-NEXT: stw 9, 12(3)
; CHECK-NEXT: stw 8, 4(3)
; CHECK-NEXT: bdnz .LBB0_1
; CHECK-NEXT: # %bb.2: # %for.end
@@ -77,11 +79,11 @@ for.end: ; preds = %for.body
define void @foo2(ptr %a, ptr readonly %b, ptr readonly %c) #0 {
; CHECK-LABEL: foo2:
; CHECK: # %bb.0: # %entry
-; CHECK-NEXT: stwu 1, -48(1)
-; CHECK-NEXT: stw 30, 40(1) # 4-byte Folded Spill
+; CHECK-NEXT: stwu 1, -64(1)
+; CHECK-NEXT: stw 30, 56(1) # 4-byte Folded Spill
; CHECK-NEXT: li 6, 2048
; CHECK-NEXT: mtctr 6
-; CHECK-NEXT: addi 6, 1, 24
+; CHECK-NEXT: addi 6, 1, 32
; CHECK-NEXT: .LBB1_1: # %for.body
; CHECK-NEXT: #
; CHECK-NEXT: lwz 7, 0(4)
@@ -89,18 +91,18 @@ define void @foo2(ptr %a, ptr readonly %b, ptr readonly %c) #0 {
; CHECK-NEXT: lwz 11, 12(5)
; CHECK-NEXT: lwz 9, 8(4)
; CHECK-NEXT: lwz 10, 12(4)
-; CHECK-NEXT: stw 8, 28(1)
+; CHECK-NEXT: stw 8, 36(1)
; CHECK-NEXT: rlwinm 8, 11, 29, 28, 29
-; CHECK-NEXT: stw 7, 24(1)
+; CHECK-NEXT: stw 7, 32(1)
; CHECK-NEXT: srawi 7, 7, 31
-; CHECK-NEXT: stw 10, 36(1)
+; CHECK-NEXT: stw 10, 44(1)
; CHECK-NEXT: clrlwi 10, 11, 27
-; CHECK-NEXT: stw 9, 32(1)
+; CHECK-NEXT: stw 9, 40(1)
; CHECK-NEXT: subfic 12, 10, 32
+; CHECK-NEXT: stw 7, 28(1)
+; CHECK-NEXT: stw 7, 24(1)
; CHECK-NEXT: stw 7, 20(1)
; CHECK-NEXT: stw 7, 16(1)
-; CHECK-NEXT: stw 7, 12(1)
-; CHECK-NEXT: stw 7, 8(1)
; CHECK-NEXT: sub 7, 6, 8
; CHECK-NEXT: lwz 8, 4(7)
; CHECK-NEXT: lwz 9, 0(7)
@@ -122,8 +124,8 @@ define void @foo2(ptr %a, ptr readonly %b, ptr readonly %c) #0 {
; CHECK-NEXT: stw 7, 4(3)
; CHECK-NEXT: bdnz .LBB1_1
; CHECK-NEXT: # %bb.2: # %for.end
-; CHECK-NEXT: lwz 30, 40(1) # 4-byte Folded Reload
-; CHECK-NEXT: addi 1, 1, 48
+; CHECK-NEXT: lwz 30, 56(1) # 4-byte Folded Reload
+; CHECK-NEXT: addi 1, 1, 64
; CHECK-NEXT: blr
entry:
br label %for.body
diff --git a/llvm/test/CodeGen/PowerPC/pr59074.ll b/llvm/test/CodeGen/PowerPC/pr59074.ll
index f4ea4087386d0..866d1b6f82ccc 100644
--- a/llvm/test/CodeGen/PowerPC/pr59074.ll
+++ b/llvm/test/CodeGen/PowerPC/pr59074.ll
@@ -27,8 +27,8 @@ define void @pr59074(ptr %0) {
; LE32-NEXT: .cfi_def_cfa_offset 80
; LE32-NEXT: lwz 4, 0(3)
; LE32-NEXT: xxlxor 0, 0, 0
-; LE32-NEXT: addi 5, 1, 20
; LE32-NEXT: li 6, 0
+; LE32-NEXT: addi 5, 1, 20
; LE32-NEXT: li 7, 12
; LE32-NEXT: addi 8, 1, 16
; LE32-NEXT: xxswapd 0, 0
@@ -37,26 +37,27 @@ define void @pr59074(ptr %0) {
; LE32-NEXT: stw 6, 40(1)
; LE32-NEXT: stw 6, 36(1)
; LE32-NEXT: stw 7, 16(1)
-; LE32-NEXT: rlwinm 9, 4, 29, 28, 29
+; LE32-NEXT: rlwinm 6, 4, 29, 28, 29
; LE32-NEXT: stxvd2x 0, 0, 5
+; LE32-NEXT: lwzx 5, 8, 6
+; LE32-NEXT: rlwimi 8, 4, 29, 28, 29
; LE32-NEXT: clrlwi 4, 4, 27
-; LE32-NEXT: lwzux 5, 9, 8
-; LE32-NEXT: lwz 6, 8(9)
-; LE32-NEXT: lwz 7, 4(9)
-; LE32-NEXT: lwz 8, 12(9)
+; LE32-NEXT: lwz 6, 8(8)
+; LE32-NEXT: lwz 7, 4(8)
+; LE32-NEXT: lwz 8, 12(8)
; LE32-NEXT: xori 9, 4, 31
; LE32-NEXT: subfic 11, 4, 32
; LE32-NEXT: srw 5, 5, 4
; LE32-NEXT: slwi 10, 6, 1
-; LE32-NEXT: srw 6, 6, 4
-; LE32-NEXT: slw 9, 10, 9
-; LE32-NEXT: srw 10, 7, 4
+; LE32-NEXT: srw 12, 7, 4
; LE32-NEXT: slw 7, 7, 11
+; LE32-NEXT: srw 6, 6, 4
; LE32-NEXT: slw 11, 8, 11
; LE32-NEXT: srw 4, 8, 4
+; LE32-NEXT: slw 9, 10, 9
; LE32-NEXT: or 5, 7, 5
; LE32-NEXT: or 6, 11, 6
-; LE32-NEXT: or 7, 10, 9
+; LE32-NEXT: or 7, 12, 9
; LE32-NEXT: stw 4, 12(3)
; LE32-NEXT: stw 6, 8(3)
; LE32-NEXT: stw 5, 0(3)
diff --git a/llvm/test/CodeGen/PowerPC/wide-scalar-shift-by-byte-multiple-legalization.ll b/llvm/test/CodeGen/PowerPC/wide-scalar-shift-by-byte-multiple-legalization.ll
index 6e66f44663295..1a3ca8fe25978 100644
--- a/llvm/test/CodeGen/PowerPC/wide-scalar-shift-by-byte-multiple-legalization.ll
+++ b/llvm/test/CodeGen/PowerPC/wide-scalar-shift-by-byte-multiple-legalization.ll
@@ -405,25 +405,26 @@ define void @shl_16bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
; LE-32BIT-NEXT: stw 9, 24(1)
; LE-32BIT-NEXT: rlwinm 4, 4, 3, 27, 28
; LE-32BIT-NEXT: stw 8, 20(1)
-; LE-32BIT-NEXT: subfic 8, 4, 32
+; LE-32BIT-NEXT: subfic 9, 4, 32
; LE-32BIT-NEXT: stw 7, 16(1)
-; LE-32BIT-NEXT: lwzux 3, 6, 3
-; LE-32BIT-NEXT: lwz 9, 4(6)
-; LE-32BIT-NEXT: slw 3, 3, 4
-; LE-32BIT-NEXT: lwz 7, 8(6)
-; LE-32BIT-NEXT: lwz 6, 12(6)
-; LE-32BIT-NEXT: slw 11, 9, 4
-; LE-32BIT-NEXT: srw 9, 9, 8
-; LE-32BIT-NEXT: srw 10, 7, 8
-; LE-32BIT-NEXT: srw 8, 6, 8
+; LE-32BIT-NEXT: lwzx 7, 3, 6
+; LE-32BIT-NEXT: rlwimi 3, 6, 0, 28, 29
+; LE-32BIT-NEXT: lwz 6, 8(3)
+; LE-32BIT-NEXT: lwz 8, 4(3)
; LE-32BIT-NEXT: slw 7, 7, 4
-; LE-32BIT-NEXT: slw 4, 6, 4
-; LE-32BIT-NEXT: or 3, 3, 9
-; LE-32BIT-NEXT: stw 4, 12(5)
-; LE-32BIT-NEXT: or 4, 7, 8
+; LE-32BIT-NEXT: lwz 3, 12(3)
+; LE-32BIT-NEXT: srw 10, 6, 9
+; LE-32BIT-NEXT: slw 11, 8, 4
+; LE-32BIT-NEXT: srw 8, 8, 9
+; LE-32BIT-NEXT: srw 9, 3, 9
+; LE-32BIT-NEXT: slw 6, 6, 4
+; LE-32BIT-NEXT: slw 3, 3, 4
+; LE-32BIT-NEXT: stw 3, 12(5)
+; LE-32BIT-NEXT: or 3, 6, 9
+; LE-32BIT-NEXT: stw 3, 8(5)
+; LE-32BIT-NEXT: or 3, 7, 8
; LE-32BIT-NEXT: stw 3, 0(5)
; LE-32BIT-NEXT: or 3, 11, 10
-; LE-32BIT-NEXT: stw 4, 8(5)
; LE-32BIT-NEXT: stw 3, 4(5)
; LE-32BIT-NEXT: addi 1, 1, 48
; LE-32BIT-NEXT: blr
@@ -483,22 +484,23 @@ define void @shl_16bytes_wordOff(ptr %src.ptr, ptr %wordOff.ptr, ptr %dst) nounw
; LE-32BIT-NEXT: lwz 4, 12(4)
; LE-32BIT-NEXT: stw 6, 44(1)
; LE-32BIT-NEXT: stw 6, 40(1)
-; LE-32BIT-NEXT: rlwinm 4, 4, 2, 28, 29
; LE-32BIT-NEXT: stw 6, 36(1)
; LE-32BIT-NEXT: stw 6, 32(1)
+; LE-32BIT-NEXT: rlwinm 6, 4, 2, 28, 29
; LE-32BIT-NEXT: stw 3, 28(1)
; LE-32BIT-NEXT: addi 3, 1, 16
; LE-32BIT-NEXT: stw 9, 24(1)
; LE-32BIT-NEXT: stw 8, 20(1)
; LE-32BIT-NEXT: stw 7, 16(1)
-; LE-32BIT-NEXT: lwzux 3, 4, 3
-; LE-32BIT-NEXT: lwz 6, 4(4)
-; LE-32BIT-NEXT: lwz 7, 12(4)
-; LE-32BIT-NEXT: lwz 4, 8(4)
-; LE-32BIT-NEXT: stw 3, 0(5)
-; LE-32BIT-NEXT: stw 4, 8(5)
+; LE-32BIT-NEXT: lwzx 6, 3, 6
+; LE-32BIT-NEXT: rlwimi 3, 4, 2, 28, 29
+; LE-32BIT-NEXT: lwz 4, 4(3)
+; LE-32BIT-NEXT: lwz 7, 12(3)
+; LE-32BIT-NEXT: lwz 3, 8(3)
+; LE-32BIT-NEXT: stw 6, 0(5)
+; LE-32BIT-NEXT: stw 3, 8(5)
; LE-32BIT-NEXT: stw 7, 12(5)
-; LE-32BIT-NEXT: stw 6, 4(5)
+; LE-32BIT-NEXT: stw 4, 4(5)
; LE-32BIT-NEXT: addi 1, 1, 48
; LE-32BIT-NEXT: blr
%src = load i128, ptr %src.ptr, align 1
diff --git a/llvm/test/CodeGen/PowerPC/wide-scalar-shift-legalization.ll b/llvm/test/CodeGen/PowerPC/wide-scalar-shift-legalization.ll
index e1731ddfd9287..5fe7fca5b6e41 100644
--- a/llvm/test/CodeGen/PowerPC/wide-scalar-shift-legalization.ll
+++ b/llvm/test/CodeGen/PowerPC/wide-scalar-shift-legalization.ll
@@ -304,27 +304,28 @@ define void @shl_16bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
; LE-32BIT-NEXT: stw 3, 28(1)
; LE-32BIT-NEXT: addi 3, 1, 16
; LE-32BIT-NEXT: stw 9, 24(1)
-; LE-32BIT-NEXT: clrlwi 4, 4, 27
; LE-32BIT-NEXT: stw 8, 20(1)
-; LE-32BIT-NEXT: subfic 8, 4, 32
; LE-32BIT-NEXT: stw 7, 16(1)
-; LE-32BIT-NEXT: lwzux 3, 6, 3
-; LE-32BIT-NEXT: lwz 9, 4(6)
-; LE-32BIT-NEXT: slw 3, 3, 4
-; LE-32BIT-NEXT: lwz 7, 8(6)
-; LE-32BIT-NEXT: lwz 6, 12(6)
-; LE-32BIT-NEXT: slw 11, 9, 4
-; LE-32BIT-NEXT: srw 9, 9, 8
-; LE-32BIT-NEXT: srw 10, 7, 8
-; LE-32BIT-NEXT: srw 8, 6, 8
+; LE-32BIT-NEXT: lwzx 6, 3, 6
+; LE-32BIT-NEXT: rlwimi 3, 4, 29, 28, 29
+; LE-32BIT-NEXT: lwz 7, 8(3)
+; LE-32BIT-NEXT: clrlwi 4, 4, 27
+; LE-32BIT-NEXT: lwz 8, 4(3)
+; LE-32BIT-NEXT: subfic 9, 4, 32
+; LE-32BIT-NEXT: lwz 3, 12(3)
+; LE-32BIT-NEXT: srw 10, 7, 9
+; LE-32BIT-NEXT: slw 11, 8, 4
+; LE-32BIT-NEXT: srw 8, 8, 9
+; LE-32BIT-NEXT: srw 9, 3, 9
; LE-32BIT-NEXT: slw 7, 7, 4
-; LE-32BIT-NEXT: slw 4, 6, 4
-; LE-32BIT-NEXT: or 3, 3, 9
-; LE-32BIT-NEXT: stw 4, 12(5)
-; LE-32BIT-NEXT: or 4, 7, 8
+; LE-32BIT-NEXT: slw 3, 3, 4
+; LE-32BIT-NEXT: slw 6, 6, 4
+; LE-32BIT-NEXT: stw 3, 12(5)
+; LE-32BIT-NEXT: or 3, 7, 9
+; LE-32BIT-NEXT: stw 3, 8(5)
+; LE-32BIT-NEXT: or 3, 6, 8
; LE-32BIT-NEXT: stw 3, 0(5)
; LE-32BIT-NEXT: or 3, 11, 10
-; LE-32BIT-NEXT: stw 4, 8(5)
; LE-32BIT-NEXT: stw 3, 4(5)
; LE-32BIT-NEXT: addi 1, 1, 48
; LE-32BIT-NEXT: blr
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/irtranslator/calling-conv-ilp32-ilp32f-ilp32d-common.ll b/llvm/test/CodeGen/RISCV/GlobalISel/irtranslator/calling-conv-ilp32-ilp32f-ilp32d-common.ll
index cc73e19f5b60b..c8d3ea8c085fd 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/irtranslator/calling-conv-ilp32-ilp32f-ilp32d-common.ll
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/irtranslator/calling-conv-ilp32-ilp32f-ilp32d-common.ll
@@ -104,10 +104,10 @@ define i64 @callee_128i_indirect_reference_in_stack(i64 %x1, i64 %x2, i64 %x3, i
; RV32I-NEXT: [[MV3:%[0-9]+]]:_(s64) = G_MERGE_VALUES [[COPY6]](s32), [[COPY7]](s32)
; RV32I-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %fixed-stack.1
; RV32I-NEXT: [[LOAD:%[0-9]+]]:_(p0) = G_LOAD [[FRAME_INDEX]](p0) :: (load (p0) from %fixed-stack.1, align 16)
- ; RV32I-NEXT: [[LOAD1:%[0-9]+]]:_(s128) = G_LOAD [[LOAD]](p0) :: (load (s128), align 8)
+ ; RV32I-NEXT: [[LOAD1:%[0-9]+]]:_(s128) = G_LOAD [[LOAD]](p0) :: (load (s128))
; RV32I-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %fixed-stack.0
; RV32I-NEXT: [[LOAD2:%[0-9]+]]:_(p0) = G_LOAD [[FRAME_INDEX1]](p0) :: (load (p0) from %fixed-stack.0)
- ; RV32I-NEXT: [[LOAD3:%[0-9]+]]:_(s128) = G_LOAD [[LOAD2]](p0) :: (load (s128), align 8)
+ ; RV32I-NEXT: [[LOAD3:%[0-9]+]]:_(s128) = G_LOAD [[LOAD2]](p0) :: (load (s128))
; RV32I-NEXT: [[ADD:%[0-9]+]]:_(s128) = G_ADD [[LOAD1]], [[LOAD3]]
; RV32I-NEXT: [[TRUNC:%[0-9]+]]:_(s64) = G_TRUNC [[ADD]](s128)
; RV32I-NEXT: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[TRUNC]](s64)
@@ -131,13 +131,13 @@ define i32 @callee_128i_indirect_refernce_in_stack( ) {
; ILP32-NEXT: [[UV4:%[0-9]+]]:_(s32), [[UV5:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
; ILP32-NEXT: [[UV6:%[0-9]+]]:_(s32), [[UV7:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
; ILP32-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32-NEXT: [[COPY:%[0-9]+]]:_(p0) = COPY $x2
; ILP32-NEXT: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
; ILP32-NEXT: [[PTR_ADD:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C3]](s32)
; ILP32-NEXT: G_STORE [[FRAME_INDEX]](p0), [[PTR_ADD]](p0) :: (store (p0) into stack, align 16)
; ILP32-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
- ; ILP32-NEXT: G_STORE [[C2]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1, align 8)
+ ; ILP32-NEXT: G_STORE [[C2]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1)
; ILP32-NEXT: [[C4:%[0-9]+]]:_(s32) = G_CONSTANT i32 4
; ILP32-NEXT: [[PTR_ADD1:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C4]](s32)
; ILP32-NEXT: G_STORE [[FRAME_INDEX1]](p0), [[PTR_ADD1]](p0) :: (store (p0) into stack + 4)
@@ -169,13 +169,13 @@ define i32 @callee_128i_indirect_refernce_in_stack( ) {
; ILP32F-NEXT: [[UV4:%[0-9]+]]:_(s32), [[UV5:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
; ILP32F-NEXT: [[UV6:%[0-9]+]]:_(s32), [[UV7:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
; ILP32F-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32F-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32F-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32F-NEXT: [[COPY:%[0-9]+]]:_(p0) = COPY $x2
; ILP32F-NEXT: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
; ILP32F-NEXT: [[PTR_ADD:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C3]](s32)
; ILP32F-NEXT: G_STORE [[FRAME_INDEX]](p0), [[PTR_ADD]](p0) :: (store (p0) into stack, align 16)
; ILP32F-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
- ; ILP32F-NEXT: G_STORE [[C2]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1, align 8)
+ ; ILP32F-NEXT: G_STORE [[C2]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1)
; ILP32F-NEXT: [[C4:%[0-9]+]]:_(s32) = G_CONSTANT i32 4
; ILP32F-NEXT: [[PTR_ADD1:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C4]](s32)
; ILP32F-NEXT: G_STORE [[FRAME_INDEX1]](p0), [[PTR_ADD1]](p0) :: (store (p0) into stack + 4)
@@ -207,13 +207,13 @@ define i32 @callee_128i_indirect_refernce_in_stack( ) {
; ILP32D-NEXT: [[UV4:%[0-9]+]]:_(s32), [[UV5:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
; ILP32D-NEXT: [[UV6:%[0-9]+]]:_(s32), [[UV7:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
; ILP32D-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32D-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32D-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32D-NEXT: [[COPY:%[0-9]+]]:_(p0) = COPY $x2
; ILP32D-NEXT: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
; ILP32D-NEXT: [[PTR_ADD:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C3]](s32)
; ILP32D-NEXT: G_STORE [[FRAME_INDEX]](p0), [[PTR_ADD]](p0) :: (store (p0) into stack, align 16)
; ILP32D-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
- ; ILP32D-NEXT: G_STORE [[C2]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1, align 8)
+ ; ILP32D-NEXT: G_STORE [[C2]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1)
; ILP32D-NEXT: [[C4:%[0-9]+]]:_(s32) = G_CONSTANT i32 4
; ILP32D-NEXT: [[PTR_ADD1:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C4]](s32)
; ILP32D-NEXT: G_STORE [[FRAME_INDEX1]](p0), [[PTR_ADD1]](p0) :: (store (p0) into stack + 4)
@@ -277,7 +277,7 @@ define i64 @callee_128i_indirect_reference_in_stack_not_first(i64 %x0, i64 %x1,
; RV32I-NEXT: [[MV7:%[0-9]+]]:_(s64) = G_MERGE_VALUES [[LOAD6]](s32), [[LOAD7]](s32)
; RV32I-NEXT: [[FRAME_INDEX8:%[0-9]+]]:_(p0) = G_FRAME_INDEX %fixed-stack.0
; RV32I-NEXT: [[LOAD8:%[0-9]+]]:_(p0) = G_LOAD [[FRAME_INDEX8]](p0) :: (load (p0) from %fixed-stack.0, align 16)
- ; RV32I-NEXT: [[LOAD9:%[0-9]+]]:_(s128) = G_LOAD [[LOAD8]](p0) :: (load (s128), align 8)
+ ; RV32I-NEXT: [[LOAD9:%[0-9]+]]:_(s128) = G_LOAD [[LOAD8]](p0) :: (load (s128))
; RV32I-NEXT: [[TRUNC:%[0-9]+]]:_(s64) = G_TRUNC [[LOAD9]](s128)
; RV32I-NEXT: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[TRUNC]](s64)
; RV32I-NEXT: $x10 = COPY [[UV]](s32)
@@ -334,7 +334,7 @@ define i32 @caller_128i_indirect_reference_in_stack_not_first() {
; ILP32-NEXT: [[PTR_ADD7:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C16]](s32)
; ILP32-NEXT: G_STORE [[UV15]](s32), [[PTR_ADD7]](p0) :: (store (s32) into stack + 28)
; ILP32-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32-NEXT: G_STORE [[C8]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32-NEXT: G_STORE [[C8]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32-NEXT: [[C17:%[0-9]+]]:_(s32) = G_CONSTANT i32 32
; ILP32-NEXT: [[PTR_ADD8:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C17]](s32)
; ILP32-NEXT: G_STORE [[FRAME_INDEX]](p0), [[PTR_ADD8]](p0) :: (store (p0) into stack + 32, align 16)
@@ -401,7 +401,7 @@ define i32 @caller_128i_indirect_reference_in_stack_not_first() {
; ILP32F-NEXT: [[PTR_ADD7:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C16]](s32)
; ILP32F-NEXT: G_STORE [[UV15]](s32), [[PTR_ADD7]](p0) :: (store (s32) into stack + 28)
; ILP32F-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32F-NEXT: G_STORE [[C8]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32F-NEXT: G_STORE [[C8]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32F-NEXT: [[C17:%[0-9]+]]:_(s32) = G_CONSTANT i32 32
; ILP32F-NEXT: [[PTR_ADD8:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C17]](s32)
; ILP32F-NEXT: G_STORE [[FRAME_INDEX]](p0), [[PTR_ADD8]](p0) :: (store (p0) into stack + 32, align 16)
@@ -468,7 +468,7 @@ define i32 @caller_128i_indirect_reference_in_stack_not_first() {
; ILP32D-NEXT: [[PTR_ADD7:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C16]](s32)
; ILP32D-NEXT: G_STORE [[UV15]](s32), [[PTR_ADD7]](p0) :: (store (s32) into stack + 28)
; ILP32D-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32D-NEXT: G_STORE [[C8]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32D-NEXT: G_STORE [[C8]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32D-NEXT: [[C17:%[0-9]+]]:_(s32) = G_CONSTANT i32 32
; ILP32D-NEXT: [[PTR_ADD8:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C17]](s32)
; ILP32D-NEXT: G_STORE [[FRAME_INDEX]](p0), [[PTR_ADD8]](p0) :: (store (p0) into stack + 32, align 16)
@@ -500,9 +500,9 @@ define i64 @callee_128i_indirect_reference_in_regs(i128 %x, i128 %y ) {
; RV32I-NEXT: liveins: $x10, $x11
; RV32I-NEXT: {{ $}}
; RV32I-NEXT: [[COPY:%[0-9]+]]:_(p0) = COPY $x10
- ; RV32I-NEXT: [[LOAD:%[0-9]+]]:_(s128) = G_LOAD [[COPY]](p0) :: (load (s128), align 8)
+ ; RV32I-NEXT: [[LOAD:%[0-9]+]]:_(s128) = G_LOAD [[COPY]](p0) :: (load (s128))
; RV32I-NEXT: [[COPY1:%[0-9]+]]:_(p0) = COPY $x11
- ; RV32I-NEXT: [[LOAD1:%[0-9]+]]:_(s128) = G_LOAD [[COPY1]](p0) :: (load (s128), align 8)
+ ; RV32I-NEXT: [[LOAD1:%[0-9]+]]:_(s128) = G_LOAD [[COPY1]](p0) :: (load (s128))
; RV32I-NEXT: [[ADD:%[0-9]+]]:_(s128) = G_ADD [[LOAD]], [[LOAD1]]
; RV32I-NEXT: [[TRUNC:%[0-9]+]]:_(s64) = G_TRUNC [[LOAD]](s128)
; RV32I-NEXT: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[TRUNC]](s64)
@@ -521,9 +521,9 @@ define i32 @caller_128i_indirect_reference_in_regs( ) {
; ILP32-NEXT: [[C1:%[0-9]+]]:_(s128) = G_CONSTANT i128 2
; ILP32-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def $x2, implicit $x2
; ILP32-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
- ; ILP32-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1, align 8)
+ ; ILP32-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1)
; ILP32-NEXT: $x10 = COPY [[FRAME_INDEX]](p0)
; ILP32-NEXT: $x11 = COPY [[FRAME_INDEX1]](p0)
; ILP32-NEXT: PseudoCALL target-flags(riscv-call) @callee_128i_indirect_reference_in_regs, csr_ilp32_lp64, implicit-def $x1, implicit $x10, implicit $x11, implicit-def $x10, implicit-def $x11
@@ -541,9 +541,9 @@ define i32 @caller_128i_indirect_reference_in_regs( ) {
; ILP32F-NEXT: [[C1:%[0-9]+]]:_(s128) = G_CONSTANT i128 2
; ILP32F-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def $x2, implicit $x2
; ILP32F-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32F-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32F-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32F-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
- ; ILP32F-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1, align 8)
+ ; ILP32F-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1)
; ILP32F-NEXT: $x10 = COPY [[FRAME_INDEX]](p0)
; ILP32F-NEXT: $x11 = COPY [[FRAME_INDEX1]](p0)
; ILP32F-NEXT: PseudoCALL target-flags(riscv-call) @callee_128i_indirect_reference_in_regs, csr_ilp32f_lp64f, implicit-def $x1, implicit $x10, implicit $x11, implicit-def $x10, implicit-def $x11
@@ -561,9 +561,9 @@ define i32 @caller_128i_indirect_reference_in_regs( ) {
; ILP32D-NEXT: [[C1:%[0-9]+]]:_(s128) = G_CONSTANT i128 2
; ILP32D-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def $x2, implicit $x2
; ILP32D-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32D-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32D-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32D-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
- ; ILP32D-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1, align 8)
+ ; ILP32D-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1)
; ILP32D-NEXT: $x10 = COPY [[FRAME_INDEX]](p0)
; ILP32D-NEXT: $x11 = COPY [[FRAME_INDEX1]](p0)
; ILP32D-NEXT: PseudoCALL target-flags(riscv-call) @callee_128i_indirect_reference_in_regs, csr_ilp32d_lp64d, implicit-def $x1, implicit $x10, implicit $x11, implicit-def $x10, implicit-def $x11
@@ -586,9 +586,9 @@ define i64 @callee_256i_indirect_reference_in_regs(i256 %x, i256 %y ) {
; RV32I-NEXT: liveins: $x10, $x11
; RV32I-NEXT: {{ $}}
; RV32I-NEXT: [[COPY:%[0-9]+]]:_(p0) = COPY $x10
- ; RV32I-NEXT: [[LOAD:%[0-9]+]]:_(s256) = G_LOAD [[COPY]](p0) :: (load (s256), align 8)
+ ; RV32I-NEXT: [[LOAD:%[0-9]+]]:_(s256) = G_LOAD [[COPY]](p0) :: (load (s256), align 16)
; RV32I-NEXT: [[COPY1:%[0-9]+]]:_(p0) = COPY $x11
- ; RV32I-NEXT: [[LOAD1:%[0-9]+]]:_(s256) = G_LOAD [[COPY1]](p0) :: (load (s256), align 8)
+ ; RV32I-NEXT: [[LOAD1:%[0-9]+]]:_(s256) = G_LOAD [[COPY1]](p0) :: (load (s256), align 16)
; RV32I-NEXT: [[ADD:%[0-9]+]]:_(s256) = G_ADD [[LOAD]], [[LOAD1]]
; RV32I-NEXT: [[TRUNC:%[0-9]+]]:_(s64) = G_TRUNC [[LOAD]](s256)
; RV32I-NEXT: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[TRUNC]](s64)
@@ -607,9 +607,9 @@ define i32 @caller_256i_indirect_reference_in_regs( ) {
; ILP32-NEXT: [[C1:%[0-9]+]]:_(s256) = G_CONSTANT i256 2
; ILP32-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def $x2, implicit $x2
; ILP32-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32-NEXT: G_STORE [[C]](s256), [[FRAME_INDEX]](p0) :: (store (s256) into %stack.0, align 8)
+ ; ILP32-NEXT: G_STORE [[C]](s256), [[FRAME_INDEX]](p0) :: (store (s256) into %stack.0, align 16)
; ILP32-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
- ; ILP32-NEXT: G_STORE [[C1]](s256), [[FRAME_INDEX1]](p0) :: (store (s256) into %stack.1, align 8)
+ ; ILP32-NEXT: G_STORE [[C1]](s256), [[FRAME_INDEX1]](p0) :: (store (s256) into %stack.1, align 16)
; ILP32-NEXT: $x10 = COPY [[FRAME_INDEX]](p0)
; ILP32-NEXT: $x11 = COPY [[FRAME_INDEX1]](p0)
; ILP32-NEXT: PseudoCALL target-flags(riscv-call) @callee_256i_indirect_reference_in_regs, csr_ilp32_lp64, implicit-def $x1, implicit $x10, implicit $x11, implicit-def $x10, implicit-def $x11
@@ -627,9 +627,9 @@ define i32 @caller_256i_indirect_reference_in_regs( ) {
; ILP32F-NEXT: [[C1:%[0-9]+]]:_(s256) = G_CONSTANT i256 2
; ILP32F-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def $x2, implicit $x2
; ILP32F-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32F-NEXT: G_STORE [[C]](s256), [[FRAME_INDEX]](p0) :: (store (s256) into %stack.0, align 8)
+ ; ILP32F-NEXT: G_STORE [[C]](s256), [[FRAME_INDEX]](p0) :: (store (s256) into %stack.0, align 16)
; ILP32F-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
- ; ILP32F-NEXT: G_STORE [[C1]](s256), [[FRAME_INDEX1]](p0) :: (store (s256) into %stack.1, align 8)
+ ; ILP32F-NEXT: G_STORE [[C1]](s256), [[FRAME_INDEX1]](p0) :: (store (s256) into %stack.1, align 16)
; ILP32F-NEXT: $x10 = COPY [[FRAME_INDEX]](p0)
; ILP32F-NEXT: $x11 = COPY [[FRAME_INDEX1]](p0)
; ILP32F-NEXT: PseudoCALL target-flags(riscv-call) @callee_256i_indirect_reference_in_regs, csr_ilp32f_lp64f, implicit-def $x1, implicit $x10, implicit $x11, implicit-def $x10, implicit-def $x11
@@ -647,9 +647,9 @@ define i32 @caller_256i_indirect_reference_in_regs( ) {
; ILP32D-NEXT: [[C1:%[0-9]+]]:_(s256) = G_CONSTANT i256 2
; ILP32D-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def $x2, implicit $x2
; ILP32D-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32D-NEXT: G_STORE [[C]](s256), [[FRAME_INDEX]](p0) :: (store (s256) into %stack.0, align 8)
+ ; ILP32D-NEXT: G_STORE [[C]](s256), [[FRAME_INDEX]](p0) :: (store (s256) into %stack.0, align 16)
; ILP32D-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
- ; ILP32D-NEXT: G_STORE [[C1]](s256), [[FRAME_INDEX1]](p0) :: (store (s256) into %stack.1, align 8)
+ ; ILP32D-NEXT: G_STORE [[C1]](s256), [[FRAME_INDEX1]](p0) :: (store (s256) into %stack.1, align 16)
; ILP32D-NEXT: $x10 = COPY [[FRAME_INDEX]](p0)
; ILP32D-NEXT: $x11 = COPY [[FRAME_INDEX1]](p0)
; ILP32D-NEXT: PseudoCALL target-flags(riscv-call) @callee_256i_indirect_reference_in_regs, csr_ilp32d_lp64d, implicit-def $x1, implicit $x10, implicit $x11, implicit-def $x10, implicit-def $x11
@@ -834,7 +834,7 @@ define i32 @callee_large_scalars(i128 %a, fp128 %b) nounwind {
; RV32I-NEXT: liveins: $x10, $x11
; RV32I-NEXT: {{ $}}
; RV32I-NEXT: [[COPY:%[0-9]+]]:_(p0) = COPY $x10
- ; RV32I-NEXT: [[LOAD:%[0-9]+]]:_(s128) = G_LOAD [[COPY]](p0) :: (load (s128), align 8)
+ ; RV32I-NEXT: [[LOAD:%[0-9]+]]:_(s128) = G_LOAD [[COPY]](p0) :: (load (s128))
; RV32I-NEXT: [[COPY1:%[0-9]+]]:_(p0) = COPY $x11
; RV32I-NEXT: [[LOAD1:%[0-9]+]]:_(s128) = G_LOAD [[COPY1]](p0) :: (load (s128))
; RV32I-NEXT: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[LOAD]](s128), [[LOAD1]]
@@ -854,7 +854,7 @@ define i32 @caller_large_scalars() nounwind {
; ILP32-NEXT: [[C1:%[0-9]+]]:_(s128) = G_FCONSTANT fp128 +inf
; ILP32-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def $x2, implicit $x2
; ILP32-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
; ILP32-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1)
; ILP32-NEXT: $x10 = COPY [[FRAME_INDEX]](p0)
@@ -871,7 +871,7 @@ define i32 @caller_large_scalars() nounwind {
; ILP32F-NEXT: [[C1:%[0-9]+]]:_(s128) = G_FCONSTANT fp128 +inf
; ILP32F-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def $x2, implicit $x2
; ILP32F-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32F-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32F-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32F-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
; ILP32F-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1)
; ILP32F-NEXT: $x10 = COPY [[FRAME_INDEX]](p0)
@@ -888,7 +888,7 @@ define i32 @caller_large_scalars() nounwind {
; ILP32D-NEXT: [[C1:%[0-9]+]]:_(s128) = G_FCONSTANT fp128 +inf
; ILP32D-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def $x2, implicit $x2
; ILP32D-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32D-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32D-NEXT: G_STORE [[C]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32D-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.1
; ILP32D-NEXT: G_STORE [[C1]](s128), [[FRAME_INDEX1]](p0) :: (store (s128) into %stack.1)
; ILP32D-NEXT: $x10 = COPY [[FRAME_INDEX]](p0)
@@ -919,7 +919,7 @@ define i32 @callee_large_scalars_exhausted_regs(i32 %a, i32 %b, i32 %c, i32 %d,
; RV32I-NEXT: [[COPY5:%[0-9]+]]:_(s32) = COPY $x15
; RV32I-NEXT: [[COPY6:%[0-9]+]]:_(s32) = COPY $x16
; RV32I-NEXT: [[COPY7:%[0-9]+]]:_(p0) = COPY $x17
- ; RV32I-NEXT: [[LOAD:%[0-9]+]]:_(s128) = G_LOAD [[COPY7]](p0) :: (load (s128), align 8)
+ ; RV32I-NEXT: [[LOAD:%[0-9]+]]:_(s128) = G_LOAD [[COPY7]](p0) :: (load (s128))
; RV32I-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %fixed-stack.1
; RV32I-NEXT: [[LOAD1:%[0-9]+]]:_(s32) = G_LOAD [[FRAME_INDEX]](p0) :: (load (s32) from %fixed-stack.1, align 16)
; RV32I-NEXT: [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %fixed-stack.0
@@ -950,7 +950,7 @@ define i32 @caller_large_scalars_exhausted_regs() nounwind {
; ILP32-NEXT: [[C9:%[0-9]+]]:_(s128) = G_FCONSTANT fp128 +inf
; ILP32-NEXT: ADJCALLSTACKDOWN 8, 0, implicit-def $x2, implicit $x2
; ILP32-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32-NEXT: G_STORE [[C7]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32-NEXT: G_STORE [[C7]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32-NEXT: [[COPY:%[0-9]+]]:_(p0) = COPY $x2
; ILP32-NEXT: [[C10:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
; ILP32-NEXT: [[PTR_ADD:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C10]](s32)
@@ -988,7 +988,7 @@ define i32 @caller_large_scalars_exhausted_regs() nounwind {
; ILP32F-NEXT: [[C9:%[0-9]+]]:_(s128) = G_FCONSTANT fp128 +inf
; ILP32F-NEXT: ADJCALLSTACKDOWN 8, 0, implicit-def $x2, implicit $x2
; ILP32F-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32F-NEXT: G_STORE [[C7]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32F-NEXT: G_STORE [[C7]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32F-NEXT: [[COPY:%[0-9]+]]:_(p0) = COPY $x2
; ILP32F-NEXT: [[C10:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
; ILP32F-NEXT: [[PTR_ADD:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C10]](s32)
@@ -1026,7 +1026,7 @@ define i32 @caller_large_scalars_exhausted_regs() nounwind {
; ILP32D-NEXT: [[C9:%[0-9]+]]:_(s128) = G_FCONSTANT fp128 +inf
; ILP32D-NEXT: ADJCALLSTACKDOWN 8, 0, implicit-def $x2, implicit $x2
; ILP32D-NEXT: [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %stack.0
- ; ILP32D-NEXT: G_STORE [[C7]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0, align 8)
+ ; ILP32D-NEXT: G_STORE [[C7]](s128), [[FRAME_INDEX]](p0) :: (store (s128) into %stack.0)
; ILP32D-NEXT: [[COPY:%[0-9]+]]:_(p0) = COPY $x2
; ILP32D-NEXT: [[C10:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
; ILP32D-NEXT: [[PTR_ADD:%[0-9]+]]:_(p0) = G_PTR_ADD [[COPY]], [[C10]](s32)
diff --git a/llvm/test/CodeGen/RISCV/bitint-fp-conv-200.ll b/llvm/test/CodeGen/RISCV/bitint-fp-conv-200.ll
index 068eb0cb14bfc..59161dd6b1f30 100644
--- a/llvm/test/CodeGen/RISCV/bitint-fp-conv-200.ll
+++ b/llvm/test/CodeGen/RISCV/bitint-fp-conv-200.ll
@@ -1178,20 +1178,20 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
;
; RV32-LABEL: test_float_to_bitint_200:
; RV32: # %bb.0: # %fp-to-i-entry
-; RV32-NEXT: addi sp, sp, -256
-; RV32-NEXT: sw ra, 252(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s0, 248(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s1, 244(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s2, 240(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s3, 236(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s4, 232(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s5, 228(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s6, 224(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s7, 220(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s8, 216(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s9, 212(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s10, 208(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s11, 204(sp) # 4-byte Folded Spill
+; RV32-NEXT: addi sp, sp, -272
+; RV32-NEXT: sw ra, 268(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s0, 264(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s1, 260(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s2, 256(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s3, 252(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s4, 248(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s5, 244(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s6, 240(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s7, 236(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s8, 232(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s9, 228(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s10, 224(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s11, 220(sp) # 4-byte Folded Spill
; RV32-NEXT: li a3, 127
; RV32-NEXT: lw a2, 0(a0)
; RV32-NEXT: slli a0, a2, 1
@@ -1211,7 +1211,7 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: srai s1, a2, 31
; RV32-NEXT: slli a2, a2, 9
; RV32-NEXT: zext.b a1, s1
-; RV32-NEXT: sw a1, 120(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 128(sp) # 4-byte Folded Spill
; RV32-NEXT: srli a2, a2, 9
; RV32-NEXT: lui a1, 2048
; RV32-NEXT: li a3, 149
@@ -1227,7 +1227,7 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: mv a2, s10
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: sw a0, 116(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 124(sp) # 4-byte Folded Spill
; RV32-NEXT: mv s2, a1
; RV32-NEXT: li a0, 0
; RV32-NEXT: li a1, 0
@@ -1243,7 +1243,7 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
; RV32-NEXT: add s2, a0, s2
-; RV32-NEXT: sw s2, 112(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s2, 120(sp) # 4-byte Folded Spill
; RV32-NEXT: sltu a0, s2, a0
; RV32-NEXT: add a0, a1, a0
; RV32-NEXT: add s2, s3, a0
@@ -1280,14 +1280,14 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: add s7, s4, s7
; RV32-NEXT: add a2, a0, s6
; RV32-NEXT: add s8, a2, s8
-; RV32-NEXT: sw s7, 108(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s7, 116(sp) # 4-byte Folded Spill
; RV32-NEXT: sltu s4, s7, s4
; RV32-NEXT: add s8, s8, s4
; RV32-NEXT: beq s8, a2, .LBB2_5
; RV32-NEXT: # %bb.4: # %fp-to-i-if-exp.small
; RV32-NEXT: sltu s4, s8, a2
; RV32-NEXT: .LBB2_5: # %fp-to-i-if-exp.small
-; RV32-NEXT: sw s8, 104(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s8, 112(sp) # 4-byte Folded Spill
; RV32-NEXT: sltu a3, s6, s3
; RV32-NEXT: sltu a0, a2, a0
; RV32-NEXT: add s6, s2, a3
@@ -1349,7 +1349,7 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: mv s1, a0
; RV32-NEXT: mv a0, s5
; RV32-NEXT: li a1, 0
-; RV32-NEXT: lw a2, 120(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 128(sp) # 4-byte Folded Reload
; RV32-NEXT: # implicit-def: $x13
; RV32-NEXT: call __muldi3
; RV32-NEXT: add a0, s1, a0
@@ -1358,25 +1358,25 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: j .LBB2_21
; RV32-NEXT: .LBB2_8: # %fp-to-i-if-exp.large
; RV32-NEXT: addi a7, a0, -150
-; RV32-NEXT: sw zero, 184(sp)
-; RV32-NEXT: sw zero, 188(sp)
; RV32-NEXT: sw zero, 192(sp)
; RV32-NEXT: sw zero, 196(sp)
-; RV32-NEXT: sw a1, 168(sp)
-; RV32-NEXT: sw zero, 172(sp)
-; RV32-NEXT: sw zero, 176(sp)
+; RV32-NEXT: sw zero, 200(sp)
+; RV32-NEXT: sw zero, 204(sp)
+; RV32-NEXT: sw a1, 176(sp)
; RV32-NEXT: sw zero, 180(sp)
+; RV32-NEXT: sw zero, 184(sp)
+; RV32-NEXT: sw zero, 188(sp)
; RV32-NEXT: srli a0, a7, 3
-; RV32-NEXT: sw zero, 152(sp)
-; RV32-NEXT: sw zero, 156(sp)
; RV32-NEXT: sw zero, 160(sp)
; RV32-NEXT: sw zero, 164(sp)
+; RV32-NEXT: sw zero, 168(sp)
+; RV32-NEXT: sw zero, 172(sp)
; RV32-NEXT: andi a0, a0, 28
-; RV32-NEXT: addi a1, sp, 168
-; RV32-NEXT: sw zero, 136(sp)
-; RV32-NEXT: sw zero, 140(sp)
+; RV32-NEXT: addi a1, sp, 176
; RV32-NEXT: sw zero, 144(sp)
; RV32-NEXT: sw zero, 148(sp)
+; RV32-NEXT: sw zero, 152(sp)
+; RV32-NEXT: sw zero, 156(sp)
; RV32-NEXT: sub a1, a1, a0
; RV32-NEXT: lw a0, 16(a1)
; RV32-NEXT: lw t0, 20(a1)
@@ -1387,19 +1387,19 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: lw a6, 12(a1)
; RV32-NEXT: xori a2, a2, 31
; RV32-NEXT: lw t1, 24(a1)
-; RV32-NEXT: sw t1, 64(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw t1, 72(sp) # 4-byte Folded Spill
; RV32-NEXT: lw a1, 28(a1)
-; RV32-NEXT: sw a1, 68(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 76(sp) # 4-byte Folded Spill
; RV32-NEXT: srli a1, a0, 1
-; RV32-NEXT: sw t0, 76(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw t0, 84(sp) # 4-byte Folded Spill
; RV32-NEXT: sll t0, t0, a7
-; RV32-NEXT: sw t0, 100(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw t0, 108(sp) # 4-byte Folded Spill
; RV32-NEXT: srl s7, a1, a2
; RV32-NEXT: sll a0, a0, a7
-; RV32-NEXT: sw a0, 108(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 116(sp) # 4-byte Folded Spill
; RV32-NEXT: srli a0, a6, 1
; RV32-NEXT: srl a0, a0, a2
-; RV32-NEXT: sw a0, 104(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 112(sp) # 4-byte Folded Spill
; RV32-NEXT: srli a0, a5, 1
; RV32-NEXT: sll a1, a6, a7
; RV32-NEXT: srl a0, a0, a2
@@ -1410,10 +1410,10 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: or s9, a1, a0
; RV32-NEXT: srli a0, a3, 1
; RV32-NEXT: sll a1, a4, a7
-; RV32-NEXT: sw a2, 72(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a2, 80(sp) # 4-byte Folded Spill
; RV32-NEXT: srl a0, a0, a2
; RV32-NEXT: or s5, a1, a0
-; RV32-NEXT: sw a7, 80(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a7, 88(sp) # 4-byte Folded Spill
; RV32-NEXT: sll s6, a3, a7
; RV32-NEXT: mv a0, s5
; RV32-NEXT: li a1, 0
@@ -1427,23 +1427,23 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: mv a2, s10
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: sw a0, 116(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 124(sp) # 4-byte Folded Spill
; RV32-NEXT: add s4, s2, a1
; RV32-NEXT: sltu a0, s4, s2
; RV32-NEXT: add s3, s3, a0
-; RV32-NEXT: sw s6, 124(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s6, 132(sp) # 4-byte Folded Spill
; RV32-NEXT: mv a0, s6
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
; RV32-NEXT: add s4, a0, s4
-; RV32-NEXT: sw s4, 112(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s4, 120(sp) # 4-byte Folded Spill
; RV32-NEXT: sltu a0, s4, a0
; RV32-NEXT: add a0, a1, a0
; RV32-NEXT: add s2, s3, a0
; RV32-NEXT: sltu s3, s2, s3
-; RV32-NEXT: sw s5, 132(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s5, 140(sp) # 4-byte Folded Spill
; RV32-NEXT: mv a0, s5
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
@@ -1460,7 +1460,7 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: call __muldi3
; RV32-NEXT: mv s4, a0
; RV32-NEXT: mv s6, a1
-; RV32-NEXT: sw s8, 128(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s8, 136(sp) # 4-byte Folded Spill
; RV32-NEXT: mv a0, s8
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s10
@@ -1469,7 +1469,7 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: mv s10, a0
; RV32-NEXT: mv s2, a1
; RV32-NEXT: add s3, a0, s6
-; RV32-NEXT: sw s9, 96(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s9, 104(sp) # 4-byte Folded Spill
; RV32-NEXT: mv a0, s9
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
@@ -1484,38 +1484,38 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: # %bb.9: # %fp-to-i-if-exp.large
; RV32-NEXT: sltu s6, s11, a2
; RV32-NEXT: .LBB2_10: # %fp-to-i-if-exp.large
-; RV32-NEXT: lw a3, 100(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 108(sp) # 4-byte Folded Reload
; RV32-NEXT: or a3, a3, s7
-; RV32-NEXT: sw a3, 92(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a3, 100(sp) # 4-byte Folded Spill
; RV32-NEXT: sltu a3, s3, s10
; RV32-NEXT: sltu a0, a2, a0
; RV32-NEXT: add a3, s2, a3
; RV32-NEXT: add a0, a1, a0
-; RV32-NEXT: lw a1, 108(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a2, 104(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a1, 116(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 112(sp) # 4-byte Folded Reload
; RV32-NEXT: or a1, a1, a2
-; RV32-NEXT: sw a1, 100(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a3, 88(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 108(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a3, 96(sp) # 4-byte Folded Spill
; RV32-NEXT: add s2, a3, a0
-; RV32-NEXT: lw a0, 128(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 136(sp) # 4-byte Folded Reload
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: sw a1, 60(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s2, 84(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a0, 52(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 68(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s2, 92(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 60(sp) # 4-byte Folded Spill
; RV32-NEXT: add s2, a0, s2
-; RV32-NEXT: sw s2, 56(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s2, 64(sp) # 4-byte Folded Spill
; RV32-NEXT: add s4, s2, s6
-; RV32-NEXT: lw a0, 132(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 140(sp) # 4-byte Folded Reload
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
; RV32-NEXT: mv s6, a0
; RV32-NEXT: mv s3, a1
-; RV32-NEXT: lw s2, 124(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s2, 132(sp) # 4-byte Folded Reload
; RV32-NEXT: mv a0, s2
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
@@ -1531,68 +1531,68 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: add s5, s8, s5
; RV32-NEXT: add a2, a0, s9
; RV32-NEXT: add s11, a2, s11
-; RV32-NEXT: sw s5, 108(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s5, 116(sp) # 4-byte Folded Spill
; RV32-NEXT: sltu s5, s5, s8
; RV32-NEXT: add s11, s11, s5
; RV32-NEXT: beq s11, a2, .LBB2_12
; RV32-NEXT: # %bb.11: # %fp-to-i-if-exp.large
; RV32-NEXT: sltu s5, s11, a2
; RV32-NEXT: .LBB2_12: # %fp-to-i-if-exp.large
-; RV32-NEXT: sw s11, 104(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s11, 112(sp) # 4-byte Folded Spill
; RV32-NEXT: sltu a3, s9, s6
; RV32-NEXT: sltu a0, a2, a0
; RV32-NEXT: add s9, s3, a3
; RV32-NEXT: add a0, a1, a0
; RV32-NEXT: add s6, s9, a0
-; RV32-NEXT: lw a0, 132(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 140(sp) # 4-byte Folded Reload
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
; RV32-NEXT: mv s8, a0
-; RV32-NEXT: sw a1, 40(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 48(sp) # 4-byte Folded Spill
; RV32-NEXT: add s3, a0, s6
; RV32-NEXT: add s2, s3, s5
; RV32-NEXT: add s7, s4, s2
-; RV32-NEXT: lw s11, 96(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s11, 104(sp) # 4-byte Folded Reload
; RV32-NEXT: mv a0, s11
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: sw a0, 48(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a1, 36(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 56(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 44(sp) # 4-byte Folded Spill
; RV32-NEXT: add s5, a0, s7
; RV32-NEXT: ori a0, s1, 1
; RV32-NEXT: li a1, 0
-; RV32-NEXT: lw a2, 100(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 108(sp) # 4-byte Folded Reload
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
; RV32-NEXT: mv s10, a0
-; RV32-NEXT: sw a1, 20(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 28(sp) # 4-byte Folded Spill
; RV32-NEXT: mv a0, s1
; RV32-NEXT: li a1, 0
-; RV32-NEXT: lw a2, 124(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 132(sp) # 4-byte Folded Reload
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
; RV32-NEXT: mv a4, a0
-; RV32-NEXT: sw a0, 8(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a1, 44(sp) # 4-byte Folded Spill
-; RV32-NEXT: lw a3, 56(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a0, 52(sp) # 4-byte Folded Reload
+; RV32-NEXT: sw a0, 16(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 52(sp) # 4-byte Folded Spill
+; RV32-NEXT: lw a3, 64(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 60(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a0, a3, a0
; RV32-NEXT: add a4, a4, s10
-; RV32-NEXT: sw a4, 4(sp) # 4-byte Folded Spill
-; RV32-NEXT: lw a1, 88(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a2, 84(sp) # 4-byte Folded Reload
+; RV32-NEXT: sw a4, 12(sp) # 4-byte Folded Spill
+; RV32-NEXT: lw a1, 96(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 92(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a1, a2, a1
; RV32-NEXT: add s10, s5, a4
-; RV32-NEXT: lw a2, 60(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 68(sp) # 4-byte Folded Reload
; RV32-NEXT: add a1, a2, a1
; RV32-NEXT: sltu a2, s4, a3
; RV32-NEXT: add a0, a0, a2
; RV32-NEXT: sltu a2, s6, s9
-; RV32-NEXT: lw a3, 40(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 48(sp) # 4-byte Folded Reload
; RV32-NEXT: add a2, a3, a2
; RV32-NEXT: sltu a3, s3, s8
; RV32-NEXT: sltu a4, s2, s3
@@ -1603,121 +1603,121 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: mv s4, a0
; RV32-NEXT: add a2, a0, a2
; RV32-NEXT: sltu s7, s10, s5
-; RV32-NEXT: sw s6, 84(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s6, 92(sp) # 4-byte Folded Spill
; RV32-NEXT: add s6, a2, s6
-; RV32-NEXT: lw a0, 128(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 136(sp) # 4-byte Folded Reload
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: sw a1, 40(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a0, 32(sp) # 4-byte Folded Spill
-; RV32-NEXT: lw s3, 36(sp) # 4-byte Folded Reload
+; RV32-NEXT: sw a1, 48(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 40(sp) # 4-byte Folded Spill
+; RV32-NEXT: lw s3, 44(sp) # 4-byte Folded Reload
; RV32-NEXT: add s3, a0, s3
; RV32-NEXT: mv a0, s11
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: sw a1, 16(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s3, 24(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a0, 12(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 24(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s3, 32(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 20(sp) # 4-byte Folded Spill
; RV32-NEXT: add s3, a0, s3
-; RV32-NEXT: lw a0, 48(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 56(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu s5, s5, a0
; RV32-NEXT: add s2, s3, s5
; RV32-NEXT: add s2, s2, s6
; RV32-NEXT: mv a0, s1
; RV32-NEXT: li a1, 0
-; RV32-NEXT: lw a2, 132(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 140(sp) # 4-byte Folded Reload
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
; RV32-NEXT: mv s9, a0
-; RV32-NEXT: sw a1, 60(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 68(sp) # 4-byte Folded Spill
; RV32-NEXT: mv a0, s1
; RV32-NEXT: li a1, 0
-; RV32-NEXT: lw a2, 124(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 132(sp) # 4-byte Folded Reload
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: sw a1, 52(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a0, 48(sp) # 4-byte Folded Spill
-; RV32-NEXT: lw a1, 44(sp) # 4-byte Folded Reload
+; RV32-NEXT: sw a1, 60(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 56(sp) # 4-byte Folded Spill
+; RV32-NEXT: lw a1, 52(sp) # 4-byte Folded Reload
; RV32-NEXT: add a0, a0, a1
-; RV32-NEXT: sw s9, 56(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a0, 44(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s9, 64(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 52(sp) # 4-byte Folded Spill
; RV32-NEXT: add s9, s9, a0
; RV32-NEXT: mv a0, s1
; RV32-NEXT: li a1, 0
-; RV32-NEXT: lw a2, 100(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 108(sp) # 4-byte Folded Reload
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: sw a1, 36(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a0, 28(sp) # 4-byte Folded Spill
-; RV32-NEXT: lw s8, 20(sp) # 4-byte Folded Reload
+; RV32-NEXT: sw a1, 44(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 36(sp) # 4-byte Folded Spill
+; RV32-NEXT: lw s8, 28(sp) # 4-byte Folded Reload
; RV32-NEXT: add s8, a0, s8
; RV32-NEXT: ori a0, s1, 1
; RV32-NEXT: li a1, 0
-; RV32-NEXT: lw a2, 92(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 100(sp) # 4-byte Folded Reload
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: lw a2, 8(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a3, 4(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 16(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 12(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a2, a3, a2
-; RV32-NEXT: sw s8, 20(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a0, 8(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s8, 28(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 16(sp) # 4-byte Folded Spill
; RV32-NEXT: add a0, a0, s8
-; RV32-NEXT: sw a0, 4(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 12(sp) # 4-byte Folded Spill
; RV32-NEXT: add s8, s9, a0
; RV32-NEXT: mv a6, a2
; RV32-NEXT: add s8, s8, a2
-; RV32-NEXT: sw s7, 88(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s7, 96(sp) # 4-byte Folded Spill
; RV32-NEXT: add s11, s8, s7
; RV32-NEXT: add s11, s2, s11
-; RV32-NEXT: sw a1, 0(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a1, 8(sp) # 4-byte Folded Spill
; RV32-NEXT: beq s11, s2, .LBB2_14
; RV32-NEXT: # %bb.13: # %fp-to-i-if-exp.large
; RV32-NEXT: sltu a0, s11, s2
-; RV32-NEXT: sw a0, 88(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a0, 96(sp) # 4-byte Folded Spill
; RV32-NEXT: .LBB2_14: # %fp-to-i-if-exp.large
-; RV32-NEXT: lw a0, 76(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 84(sp) # 4-byte Folded Reload
; RV32-NEXT: srli a0, a0, 1
-; RV32-NEXT: lw a2, 64(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 72(sp) # 4-byte Folded Reload
; RV32-NEXT: srli a1, a2, 1
; RV32-NEXT: beq s2, s3, .LBB2_16
; RV32-NEXT: # %bb.15: # %fp-to-i-if-exp.large
; RV32-NEXT: sltu s5, s2, s3
; RV32-NEXT: .LBB2_16: # %fp-to-i-if-exp.large
-; RV32-NEXT: lw a4, 80(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 88(sp) # 4-byte Folded Reload
; RV32-NEXT: sll a2, a2, a4
-; RV32-NEXT: lw a5, 72(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a5, 80(sp) # 4-byte Folded Reload
; RV32-NEXT: srl a3, a0, a5
-; RV32-NEXT: lw a0, 68(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 76(sp) # 4-byte Folded Reload
; RV32-NEXT: sll a0, a0, a4
; RV32-NEXT: srl a1, a1, a5
; RV32-NEXT: mv s7, a6
; RV32-NEXT: beq s6, s4, .LBB2_18
; RV32-NEXT: # %bb.17: # %fp-to-i-if-exp.large
; RV32-NEXT: sltu a4, s6, s4
-; RV32-NEXT: sw a4, 84(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a4, 92(sp) # 4-byte Folded Spill
; RV32-NEXT: .LBB2_18: # %fp-to-i-if-exp.large
; RV32-NEXT: or s2, a2, a3
-; RV32-NEXT: lw a2, 32(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a3, 24(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 40(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 32(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a2, a3, a2
-; RV32-NEXT: lw a3, 12(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 20(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a3, s3, a3
-; RV32-NEXT: lw a4, 40(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 48(sp) # 4-byte Folded Reload
; RV32-NEXT: add a2, a4, a2
-; RV32-NEXT: lw a4, 16(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 24(sp) # 4-byte Folded Reload
; RV32-NEXT: add a3, a4, a3
; RV32-NEXT: or s3, a0, a1
; RV32-NEXT: add s4, a2, a3
-; RV32-NEXT: lw a0, 128(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 136(sp) # 4-byte Folded Reload
; RV32-NEXT: li a1, 0
; RV32-NEXT: mv a2, s1
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: lw a1, 84(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a1, 92(sp) # 4-byte Folded Reload
; RV32-NEXT: add a0, a0, a1
; RV32-NEXT: add a0, a0, s5
; RV32-NEXT: add s5, a0, s4
@@ -1725,8 +1725,8 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: # %bb.19: # %fp-to-i-if-exp.large
; RV32-NEXT: sltu s7, s8, s9
; RV32-NEXT: .LBB2_20: # %fp-to-i-if-exp.large
-; RV32-NEXT: lw a0, 100(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s6, 92(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 108(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s6, 100(sp) # 4-byte Folded Reload
; RV32-NEXT: mv a1, s6
; RV32-NEXT: mv a2, s1
; RV32-NEXT: mv a3, s1
@@ -1737,15 +1737,15 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: ori a2, s1, 1
; RV32-NEXT: mv a3, s1
; RV32-NEXT: call __muldi3
-; RV32-NEXT: lw a1, 28(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a2, 20(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a1, 36(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 28(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a1, a2, a1
-; RV32-NEXT: lw a2, 8(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a3, 4(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 16(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 12(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a2, a3, a2
-; RV32-NEXT: lw a3, 36(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 44(sp) # 4-byte Folded Reload
; RV32-NEXT: add a1, a3, a1
-; RV32-NEXT: lw a3, 0(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 8(sp) # 4-byte Folded Reload
; RV32-NEXT: add a2, a3, a2
; RV32-NEXT: add s4, a0, s4
; RV32-NEXT: add s2, a1, a2
@@ -1756,25 +1756,25 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: call __muldi3
; RV32-NEXT: add a0, a0, s4
; RV32-NEXT: add s3, a0, s2
-; RV32-NEXT: lw a0, 96(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a1, 128(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 104(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a1, 136(sp) # 4-byte Folded Reload
; RV32-NEXT: mv a2, s1
; RV32-NEXT: mv a3, s1
; RV32-NEXT: call __muldi3
; RV32-NEXT: mv s2, a0
-; RV32-NEXT: lw a0, 124(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s6, 132(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a0, 132(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s6, 140(sp) # 4-byte Folded Reload
; RV32-NEXT: mv a1, s6
-; RV32-NEXT: lw a2, 120(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 128(sp) # 4-byte Folded Reload
; RV32-NEXT: call __muldi3
-; RV32-NEXT: lw a1, 56(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a1, 64(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a1, s9, a1
-; RV32-NEXT: lw a2, 48(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a3, 44(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 56(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 52(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a2, a3, a2
-; RV32-NEXT: lw a3, 60(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 68(sp) # 4-byte Folded Reload
; RV32-NEXT: add a1, a3, a1
-; RV32-NEXT: lw a3, 52(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 60(sp) # 4-byte Folded Reload
; RV32-NEXT: add a2, a3, a2
; RV32-NEXT: add s2, s2, a0
; RV32-NEXT: add s4, a2, a1
@@ -1783,7 +1783,7 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: mv a2, s6
; RV32-NEXT: li a3, 0
; RV32-NEXT: call __muldi3
-; RV32-NEXT: lw a1, 88(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a1, 96(sp) # 4-byte Folded Reload
; RV32-NEXT: add s5, s5, a1
; RV32-NEXT: add a0, a0, s2
; RV32-NEXT: add a0, a0, s4
@@ -1792,10 +1792,10 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: add a0, s5, a0
; RV32-NEXT: .LBB2_21: # %fp-to-i-cleanup
; RV32-NEXT: zext.b a0, a0
-; RV32-NEXT: lw a1, 116(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a2, 112(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a3, 108(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a4, 104(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a1, 124(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 120(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 116(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 112(sp) # 4-byte Folded Reload
; RV32-NEXT: .LBB2_22: # %fp-to-i-cleanup
; RV32-NEXT: sw a1, 0(s0)
; RV32-NEXT: sw a2, 4(s0)
@@ -1804,20 +1804,20 @@ define void @test_float_to_bitint_200(ptr %in, ptr %out) nounwind {
; RV32-NEXT: sw s10, 16(s0)
; RV32-NEXT: sw s11, 20(s0)
; RV32-NEXT: sb a0, 24(s0)
-; RV32-NEXT: lw ra, 252(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s0, 248(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s1, 244(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s2, 240(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s3, 236(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s4, 232(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s5, 228(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s6, 224(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s7, 220(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s8, 216(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s9, 212(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s10, 208(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s11, 204(sp) # 4-byte Folded Reload
-; RV32-NEXT: addi sp, sp, 256
+; RV32-NEXT: lw ra, 268(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s0, 264(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s1, 260(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s2, 256(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s3, 252(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s4, 248(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s5, 244(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s6, 240(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s7, 236(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s8, 232(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s9, 228(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s10, 224(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s11, 220(sp) # 4-byte Folded Reload
+; RV32-NEXT: addi sp, sp, 272
; RV32-NEXT: ret
%1 = load float, ptr %in, align 4
%2 = fptosi float %1 to i200
diff --git a/llvm/test/CodeGen/RISCV/calling-conv-ilp32-ilp32f-ilp32d-common.ll b/llvm/test/CodeGen/RISCV/calling-conv-ilp32-ilp32f-ilp32d-common.ll
index aa48be68b4b83..5719807589e52 100644
--- a/llvm/test/CodeGen/RISCV/calling-conv-ilp32-ilp32f-ilp32d-common.ll
+++ b/llvm/test/CodeGen/RISCV/calling-conv-ilp32-ilp32f-ilp32d-common.ll
@@ -251,12 +251,12 @@ define i32 @caller_large_scalars() nounwind {
; RV32I-FPELIM-NEXT: sw zero, 8(sp)
; RV32I-FPELIM-NEXT: sw a0, 12(sp)
; RV32I-FPELIM-NEXT: li a2, 1
-; RV32I-FPELIM-NEXT: addi a0, sp, 24
+; RV32I-FPELIM-NEXT: addi a0, sp, 16
; RV32I-FPELIM-NEXT: mv a1, sp
-; RV32I-FPELIM-NEXT: sw a2, 24(sp)
+; RV32I-FPELIM-NEXT: sw a2, 16(sp)
+; RV32I-FPELIM-NEXT: sw zero, 20(sp)
+; RV32I-FPELIM-NEXT: sw zero, 24(sp)
; RV32I-FPELIM-NEXT: sw zero, 28(sp)
-; RV32I-FPELIM-NEXT: sw zero, 32(sp)
-; RV32I-FPELIM-NEXT: sw zero, 36(sp)
; RV32I-FPELIM-NEXT: call callee_large_scalars
; RV32I-FPELIM-NEXT: lw ra, 44(sp) # 4-byte Folded Reload
; RV32I-FPELIM-NEXT: addi sp, sp, 48
@@ -274,12 +274,12 @@ define i32 @caller_large_scalars() nounwind {
; RV32I-WITHFP-NEXT: sw zero, 8(sp)
; RV32I-WITHFP-NEXT: sw a0, 12(sp)
; RV32I-WITHFP-NEXT: li a2, 1
-; RV32I-WITHFP-NEXT: addi a0, sp, 24
+; RV32I-WITHFP-NEXT: addi a0, sp, 16
; RV32I-WITHFP-NEXT: mv a1, sp
-; RV32I-WITHFP-NEXT: sw a2, 24(sp)
+; RV32I-WITHFP-NEXT: sw a2, 16(sp)
+; RV32I-WITHFP-NEXT: sw zero, 20(sp)
+; RV32I-WITHFP-NEXT: sw zero, 24(sp)
; RV32I-WITHFP-NEXT: sw zero, 28(sp)
-; RV32I-WITHFP-NEXT: sw zero, 32(sp)
-; RV32I-WITHFP-NEXT: sw zero, 36(sp)
; RV32I-WITHFP-NEXT: call callee_large_scalars
; RV32I-WITHFP-NEXT: lw ra, 44(sp) # 4-byte Folded Reload
; RV32I-WITHFP-NEXT: lw s0, 40(sp) # 4-byte Folded Reload
@@ -370,11 +370,11 @@ define i32 @caller_large_scalars_exhausted_regs() nounwind {
; RV32I-FPELIM-NEXT: li a4, 5
; RV32I-FPELIM-NEXT: li a5, 6
; RV32I-FPELIM-NEXT: li a6, 7
-; RV32I-FPELIM-NEXT: addi a7, sp, 40
-; RV32I-FPELIM-NEXT: sw t0, 40(sp)
+; RV32I-FPELIM-NEXT: addi a7, sp, 32
+; RV32I-FPELIM-NEXT: sw t0, 32(sp)
+; RV32I-FPELIM-NEXT: sw zero, 36(sp)
+; RV32I-FPELIM-NEXT: sw zero, 40(sp)
; RV32I-FPELIM-NEXT: sw zero, 44(sp)
-; RV32I-FPELIM-NEXT: sw zero, 48(sp)
-; RV32I-FPELIM-NEXT: sw zero, 52(sp)
; RV32I-FPELIM-NEXT: call callee_large_scalars_exhausted_regs
; RV32I-FPELIM-NEXT: lw ra, 60(sp) # 4-byte Folded Reload
; RV32I-FPELIM-NEXT: addi sp, sp, 64
@@ -403,11 +403,11 @@ define i32 @caller_large_scalars_exhausted_regs() nounwind {
; RV32I-WITHFP-NEXT: li a4, 5
; RV32I-WITHFP-NEXT: li a5, 6
; RV32I-WITHFP-NEXT: li a6, 7
-; RV32I-WITHFP-NEXT: addi a7, sp, 40
-; RV32I-WITHFP-NEXT: sw t0, 40(sp)
+; RV32I-WITHFP-NEXT: addi a7, sp, 32
+; RV32I-WITHFP-NEXT: sw t0, 32(sp)
+; RV32I-WITHFP-NEXT: sw zero, 36(sp)
+; RV32I-WITHFP-NEXT: sw zero, 40(sp)
; RV32I-WITHFP-NEXT: sw zero, 44(sp)
-; RV32I-WITHFP-NEXT: sw zero, 48(sp)
-; RV32I-WITHFP-NEXT: sw zero, 52(sp)
; RV32I-WITHFP-NEXT: call callee_large_scalars_exhausted_regs
; RV32I-WITHFP-NEXT: lw ra, 60(sp) # 4-byte Folded Reload
; RV32I-WITHFP-NEXT: lw s0, 56(sp) # 4-byte Folded Reload
@@ -428,9 +428,9 @@ define i32 @caller_mixed_scalar_libcalls(i64 %a) nounwind {
; RV32I-FPELIM-NEXT: sw ra, 28(sp) # 4-byte Folded Spill
; RV32I-FPELIM-NEXT: mv a2, a1
; RV32I-FPELIM-NEXT: mv a1, a0
-; RV32I-FPELIM-NEXT: addi a0, sp, 8
+; RV32I-FPELIM-NEXT: mv a0, sp
; RV32I-FPELIM-NEXT: call __floatditf
-; RV32I-FPELIM-NEXT: lw a0, 8(sp)
+; RV32I-FPELIM-NEXT: lw a0, 0(sp)
; RV32I-FPELIM-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; RV32I-FPELIM-NEXT: addi sp, sp, 32
; RV32I-FPELIM-NEXT: ret
@@ -443,9 +443,9 @@ define i32 @caller_mixed_scalar_libcalls(i64 %a) nounwind {
; RV32I-WITHFP-NEXT: addi s0, sp, 32
; RV32I-WITHFP-NEXT: mv a2, a1
; RV32I-WITHFP-NEXT: mv a1, a0
-; RV32I-WITHFP-NEXT: addi a0, sp, 8
+; RV32I-WITHFP-NEXT: mv a0, sp
; RV32I-WITHFP-NEXT: call __floatditf
-; RV32I-WITHFP-NEXT: lw a0, 8(sp)
+; RV32I-WITHFP-NEXT: lw a0, 0(sp)
; RV32I-WITHFP-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; RV32I-WITHFP-NEXT: lw s0, 24(sp) # 4-byte Folded Reload
; RV32I-WITHFP-NEXT: addi sp, sp, 32
diff --git a/llvm/test/CodeGen/RISCV/calling-conv-ilp32e.ll b/llvm/test/CodeGen/RISCV/calling-conv-ilp32e.ll
index c3c997aabce60..df80a2c7cd884 100644
--- a/llvm/test/CodeGen/RISCV/calling-conv-ilp32e.ll
+++ b/llvm/test/CodeGen/RISCV/calling-conv-ilp32e.ll
@@ -1509,12 +1509,12 @@ define i32 @caller_large_scalars() {
; ILP32E-FPELIM-NEXT: sw zero, 8(sp)
; ILP32E-FPELIM-NEXT: sw a0, 12(sp)
; ILP32E-FPELIM-NEXT: li a2, 1
-; ILP32E-FPELIM-NEXT: addi a0, sp, 24
+; ILP32E-FPELIM-NEXT: addi a0, sp, 16
; ILP32E-FPELIM-NEXT: mv a1, sp
-; ILP32E-FPELIM-NEXT: sw a2, 24(sp)
+; ILP32E-FPELIM-NEXT: sw a2, 16(sp)
+; ILP32E-FPELIM-NEXT: sw zero, 20(sp)
+; ILP32E-FPELIM-NEXT: sw zero, 24(sp)
; ILP32E-FPELIM-NEXT: sw zero, 28(sp)
-; ILP32E-FPELIM-NEXT: sw zero, 32(sp)
-; ILP32E-FPELIM-NEXT: sw zero, 36(sp)
; ILP32E-FPELIM-NEXT: call callee_large_scalars
; ILP32E-FPELIM-NEXT: addi sp, s0, -48
; ILP32E-FPELIM-NEXT: .cfi_def_cfa sp, 48
@@ -1543,12 +1543,12 @@ define i32 @caller_large_scalars() {
; ILP32E-WITHFP-NEXT: sw zero, 8(sp)
; ILP32E-WITHFP-NEXT: sw a0, 12(sp)
; ILP32E-WITHFP-NEXT: li a2, 1
-; ILP32E-WITHFP-NEXT: addi a0, sp, 24
+; ILP32E-WITHFP-NEXT: addi a0, sp, 16
; ILP32E-WITHFP-NEXT: mv a1, sp
-; ILP32E-WITHFP-NEXT: sw a2, 24(sp)
+; ILP32E-WITHFP-NEXT: sw a2, 16(sp)
+; ILP32E-WITHFP-NEXT: sw zero, 20(sp)
+; ILP32E-WITHFP-NEXT: sw zero, 24(sp)
; ILP32E-WITHFP-NEXT: sw zero, 28(sp)
-; ILP32E-WITHFP-NEXT: sw zero, 32(sp)
-; ILP32E-WITHFP-NEXT: sw zero, 36(sp)
; ILP32E-WITHFP-NEXT: call callee_large_scalars
; ILP32E-WITHFP-NEXT: addi sp, s0, -48
; ILP32E-WITHFP-NEXT: .cfi_def_cfa sp, 48
@@ -1577,12 +1577,12 @@ define i32 @caller_large_scalars() {
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 8(sp)
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw a0, 12(sp)
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: li a2, 1
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi a0, sp, 24
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi a0, sp, 16
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: mv a1, sp
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw a2, 24(sp)
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw a2, 16(sp)
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 20(sp)
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 24(sp)
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 28(sp)
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 32(sp)
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 36(sp)
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: call callee_large_scalars
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi sp, s0, -48
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_def_cfa sp, 48
@@ -1607,12 +1607,12 @@ define i32 @caller_large_scalars() {
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 8(sp)
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw a0, 12(sp)
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: li a2, 1
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi a0, sp, 24
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi a0, sp, 16
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: mv a1, sp
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw a2, 24(sp)
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw a2, 16(sp)
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 20(sp)
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 24(sp)
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 28(sp)
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 32(sp)
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 36(sp)
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: call callee_large_scalars
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi sp, s0, -48
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_def_cfa sp, 48
@@ -1757,7 +1757,7 @@ define i32 @caller_large_scalars_exhausted_regs() {
; ILP32E-FPELIM-NEXT: andi sp, sp, -16
; ILP32E-FPELIM-NEXT: addi a0, sp, 16
; ILP32E-FPELIM-NEXT: li a1, 9
-; ILP32E-FPELIM-NEXT: addi a2, sp, 40
+; ILP32E-FPELIM-NEXT: addi a2, sp, 32
; ILP32E-FPELIM-NEXT: li a3, 7
; ILP32E-FPELIM-NEXT: sw a3, 0(sp)
; ILP32E-FPELIM-NEXT: sw a2, 4(sp)
@@ -1775,10 +1775,10 @@ define i32 @caller_large_scalars_exhausted_regs() {
; ILP32E-FPELIM-NEXT: li a3, 4
; ILP32E-FPELIM-NEXT: li a4, 5
; ILP32E-FPELIM-NEXT: li a5, 6
-; ILP32E-FPELIM-NEXT: sw a6, 40(sp)
+; ILP32E-FPELIM-NEXT: sw a6, 32(sp)
+; ILP32E-FPELIM-NEXT: sw zero, 36(sp)
+; ILP32E-FPELIM-NEXT: sw zero, 40(sp)
; ILP32E-FPELIM-NEXT: sw zero, 44(sp)
-; ILP32E-FPELIM-NEXT: sw zero, 48(sp)
-; ILP32E-FPELIM-NEXT: sw zero, 52(sp)
; ILP32E-FPELIM-NEXT: call callee_large_scalars_exhausted_regs
; ILP32E-FPELIM-NEXT: addi sp, s0, -64
; ILP32E-FPELIM-NEXT: .cfi_def_cfa sp, 64
@@ -1803,7 +1803,7 @@ define i32 @caller_large_scalars_exhausted_regs() {
; ILP32E-WITHFP-NEXT: andi sp, sp, -16
; ILP32E-WITHFP-NEXT: addi a0, sp, 16
; ILP32E-WITHFP-NEXT: li a1, 9
-; ILP32E-WITHFP-NEXT: addi a2, sp, 40
+; ILP32E-WITHFP-NEXT: addi a2, sp, 32
; ILP32E-WITHFP-NEXT: li a3, 7
; ILP32E-WITHFP-NEXT: sw a3, 0(sp)
; ILP32E-WITHFP-NEXT: sw a2, 4(sp)
@@ -1821,10 +1821,10 @@ define i32 @caller_large_scalars_exhausted_regs() {
; ILP32E-WITHFP-NEXT: li a3, 4
; ILP32E-WITHFP-NEXT: li a4, 5
; ILP32E-WITHFP-NEXT: li a5, 6
-; ILP32E-WITHFP-NEXT: sw a6, 40(sp)
+; ILP32E-WITHFP-NEXT: sw a6, 32(sp)
+; ILP32E-WITHFP-NEXT: sw zero, 36(sp)
+; ILP32E-WITHFP-NEXT: sw zero, 40(sp)
; ILP32E-WITHFP-NEXT: sw zero, 44(sp)
-; ILP32E-WITHFP-NEXT: sw zero, 48(sp)
-; ILP32E-WITHFP-NEXT: sw zero, 52(sp)
; ILP32E-WITHFP-NEXT: call callee_large_scalars_exhausted_regs
; ILP32E-WITHFP-NEXT: addi sp, s0, -64
; ILP32E-WITHFP-NEXT: .cfi_def_cfa sp, 64
@@ -1849,7 +1849,7 @@ define i32 @caller_large_scalars_exhausted_regs() {
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: andi sp, sp, -16
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi a0, sp, 16
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: li a1, 9
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi a2, sp, 40
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi a2, sp, 32
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: li a3, 7
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw a3, 0(sp)
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw a2, 4(sp)
@@ -1867,10 +1867,10 @@ define i32 @caller_large_scalars_exhausted_regs() {
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: li a3, 4
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: li a4, 5
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: li a5, 6
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw a6, 40(sp)
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw a6, 32(sp)
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 36(sp)
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 40(sp)
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 44(sp)
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 48(sp)
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: sw zero, 52(sp)
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: call callee_large_scalars_exhausted_regs
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi sp, s0, -64
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_def_cfa sp, 64
@@ -1891,7 +1891,7 @@ define i32 @caller_large_scalars_exhausted_regs() {
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: andi sp, sp, -16
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi a0, sp, 16
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: li a1, 9
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi a2, sp, 40
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi a2, sp, 32
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: li a3, 7
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw a3, 0(sp)
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw a2, 4(sp)
@@ -1909,10 +1909,10 @@ define i32 @caller_large_scalars_exhausted_regs() {
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: li a3, 4
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: li a4, 5
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: li a5, 6
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw a6, 40(sp)
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw a6, 32(sp)
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 36(sp)
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 40(sp)
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 44(sp)
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 48(sp)
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: sw zero, 52(sp)
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: call callee_large_scalars_exhausted_regs
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi sp, s0, -64
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_def_cfa sp, 64
@@ -1930,53 +1930,53 @@ define i32 @caller_large_scalars_exhausted_regs() {
define i32 @caller_mixed_scalar_libcalls(i64 %a) {
; ILP32E-FPELIM-LABEL: caller_mixed_scalar_libcalls:
; ILP32E-FPELIM: # %bb.0:
-; ILP32E-FPELIM-NEXT: addi sp, sp, -24
-; ILP32E-FPELIM-NEXT: .cfi_def_cfa_offset 24
-; ILP32E-FPELIM-NEXT: sw ra, 20(sp) # 4-byte Folded Spill
-; ILP32E-FPELIM-NEXT: sw s0, 16(sp) # 4-byte Folded Spill
+; ILP32E-FPELIM-NEXT: addi sp, sp, -32
+; ILP32E-FPELIM-NEXT: .cfi_def_cfa_offset 32
+; ILP32E-FPELIM-NEXT: sw ra, 28(sp) # 4-byte Folded Spill
+; ILP32E-FPELIM-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; ILP32E-FPELIM-NEXT: .cfi_offset ra, -4
; ILP32E-FPELIM-NEXT: .cfi_offset s0, -8
-; ILP32E-FPELIM-NEXT: addi s0, sp, 24
+; ILP32E-FPELIM-NEXT: addi s0, sp, 32
; ILP32E-FPELIM-NEXT: .cfi_def_cfa s0, 0
-; ILP32E-FPELIM-NEXT: andi sp, sp, -8
+; ILP32E-FPELIM-NEXT: andi sp, sp, -16
; ILP32E-FPELIM-NEXT: mv a2, a1
; ILP32E-FPELIM-NEXT: mv a1, a0
; ILP32E-FPELIM-NEXT: mv a0, sp
; ILP32E-FPELIM-NEXT: call __floatditf
; ILP32E-FPELIM-NEXT: lw a0, 0(sp)
-; ILP32E-FPELIM-NEXT: addi sp, s0, -24
-; ILP32E-FPELIM-NEXT: .cfi_def_cfa sp, 24
-; ILP32E-FPELIM-NEXT: lw ra, 20(sp) # 4-byte Folded Reload
-; ILP32E-FPELIM-NEXT: lw s0, 16(sp) # 4-byte Folded Reload
+; ILP32E-FPELIM-NEXT: addi sp, s0, -32
+; ILP32E-FPELIM-NEXT: .cfi_def_cfa sp, 32
+; ILP32E-FPELIM-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
+; ILP32E-FPELIM-NEXT: lw s0, 24(sp) # 4-byte Folded Reload
; ILP32E-FPELIM-NEXT: .cfi_restore ra
; ILP32E-FPELIM-NEXT: .cfi_restore s0
-; ILP32E-FPELIM-NEXT: addi sp, sp, 24
+; ILP32E-FPELIM-NEXT: addi sp, sp, 32
; ILP32E-FPELIM-NEXT: .cfi_def_cfa_offset 0
; ILP32E-FPELIM-NEXT: ret
;
; ILP32E-WITHFP-LABEL: caller_mixed_scalar_libcalls:
; ILP32E-WITHFP: # %bb.0:
-; ILP32E-WITHFP-NEXT: addi sp, sp, -24
-; ILP32E-WITHFP-NEXT: .cfi_def_cfa_offset 24
-; ILP32E-WITHFP-NEXT: sw ra, 20(sp) # 4-byte Folded Spill
-; ILP32E-WITHFP-NEXT: sw s0, 16(sp) # 4-byte Folded Spill
+; ILP32E-WITHFP-NEXT: addi sp, sp, -32
+; ILP32E-WITHFP-NEXT: .cfi_def_cfa_offset 32
+; ILP32E-WITHFP-NEXT: sw ra, 28(sp) # 4-byte Folded Spill
+; ILP32E-WITHFP-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; ILP32E-WITHFP-NEXT: .cfi_offset ra, -4
; ILP32E-WITHFP-NEXT: .cfi_offset s0, -8
-; ILP32E-WITHFP-NEXT: addi s0, sp, 24
+; ILP32E-WITHFP-NEXT: addi s0, sp, 32
; ILP32E-WITHFP-NEXT: .cfi_def_cfa s0, 0
-; ILP32E-WITHFP-NEXT: andi sp, sp, -8
+; ILP32E-WITHFP-NEXT: andi sp, sp, -16
; ILP32E-WITHFP-NEXT: mv a2, a1
; ILP32E-WITHFP-NEXT: mv a1, a0
; ILP32E-WITHFP-NEXT: mv a0, sp
; ILP32E-WITHFP-NEXT: call __floatditf
; ILP32E-WITHFP-NEXT: lw a0, 0(sp)
-; ILP32E-WITHFP-NEXT: addi sp, s0, -24
-; ILP32E-WITHFP-NEXT: .cfi_def_cfa sp, 24
-; ILP32E-WITHFP-NEXT: lw ra, 20(sp) # 4-byte Folded Reload
-; ILP32E-WITHFP-NEXT: lw s0, 16(sp) # 4-byte Folded Reload
+; ILP32E-WITHFP-NEXT: addi sp, s0, -32
+; ILP32E-WITHFP-NEXT: .cfi_def_cfa sp, 32
+; ILP32E-WITHFP-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
+; ILP32E-WITHFP-NEXT: lw s0, 24(sp) # 4-byte Folded Reload
; ILP32E-WITHFP-NEXT: .cfi_restore ra
; ILP32E-WITHFP-NEXT: .cfi_restore s0
-; ILP32E-WITHFP-NEXT: addi sp, sp, 24
+; ILP32E-WITHFP-NEXT: addi sp, sp, 32
; ILP32E-WITHFP-NEXT: .cfi_def_cfa_offset 0
; ILP32E-WITHFP-NEXT: ret
;
@@ -1986,19 +1986,19 @@ define i32 @caller_mixed_scalar_libcalls(i64 %a) {
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_def_cfa_offset 8
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_offset ra, -4
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_offset s0, -8
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi sp, sp, -16
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_def_cfa_offset 24
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi s0, sp, 24
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi sp, sp, -24
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_def_cfa_offset 32
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi s0, sp, 32
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_def_cfa s0, 0
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: andi sp, sp, -8
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: andi sp, sp, -16
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: mv a2, a1
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: mv a1, a0
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: mv a0, sp
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: call __floatditf
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: lw a0, 0(sp)
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi sp, s0, -24
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_def_cfa sp, 24
-; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi sp, sp, 16
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi sp, s0, -32
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_def_cfa sp, 32
+; ILP32E-FPELIM-SAVE-RESTORE-NEXT: addi sp, sp, 24
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: .cfi_def_cfa_offset 8
; ILP32E-FPELIM-SAVE-RESTORE-NEXT: tail __riscv_restore_1
;
@@ -2008,19 +2008,19 @@ define i32 @caller_mixed_scalar_libcalls(i64 %a) {
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_def_cfa_offset 8
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_offset ra, -4
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_offset s0, -8
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi sp, sp, -16
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_def_cfa_offset 24
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi s0, sp, 24
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi sp, sp, -24
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_def_cfa_offset 32
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi s0, sp, 32
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_def_cfa s0, 0
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: andi sp, sp, -8
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: andi sp, sp, -16
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: mv a2, a1
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: mv a1, a0
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: mv a0, sp
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: call __floatditf
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: lw a0, 0(sp)
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi sp, s0, -24
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_def_cfa sp, 24
-; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi sp, sp, 16
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi sp, s0, -32
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_def_cfa sp, 32
+; ILP32E-WITHFP-SAVE-RESTORE-NEXT: addi sp, sp, 24
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: .cfi_def_cfa_offset 8
; ILP32E-WITHFP-SAVE-RESTORE-NEXT: tail __riscv_restore_1
%1 = sitofp i64 %a to fp128
diff --git a/llvm/test/CodeGen/RISCV/forced-atomics.ll b/llvm/test/CodeGen/RISCV/forced-atomics.ll
index a15ecc6a15878..12d9f869f2cf8 100644
--- a/llvm/test/CodeGen/RISCV/forced-atomics.ll
+++ b/llvm/test/CodeGen/RISCV/forced-atomics.ll
@@ -4525,13 +4525,13 @@ define i128 @load128(ptr %p) nounwind {
; RV32-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; RV32-NEXT: mv s0, a0
; RV32-NEXT: li a0, 16
-; RV32-NEXT: addi a2, sp, 8
+; RV32-NEXT: mv a2, sp
; RV32-NEXT: li a3, 5
; RV32-NEXT: call __atomic_load
-; RV32-NEXT: lw a0, 8(sp)
-; RV32-NEXT: lw a1, 12(sp)
-; RV32-NEXT: lw a2, 16(sp)
-; RV32-NEXT: lw a3, 20(sp)
+; RV32-NEXT: lw a0, 0(sp)
+; RV32-NEXT: lw a1, 4(sp)
+; RV32-NEXT: lw a2, 8(sp)
+; RV32-NEXT: lw a3, 12(sp)
; RV32-NEXT: sw a0, 0(s0)
; RV32-NEXT: sw a1, 4(s0)
; RV32-NEXT: sw a2, 8(s0)
@@ -4560,12 +4560,12 @@ define void @store128(ptr %p) nounwind {
; RV32-NEXT: addi sp, sp, -32
; RV32-NEXT: sw ra, 28(sp) # 4-byte Folded Spill
; RV32-NEXT: mv a1, a0
+; RV32-NEXT: sw zero, 0(sp)
+; RV32-NEXT: sw zero, 4(sp)
; RV32-NEXT: sw zero, 8(sp)
; RV32-NEXT: sw zero, 12(sp)
-; RV32-NEXT: sw zero, 16(sp)
-; RV32-NEXT: sw zero, 20(sp)
; RV32-NEXT: li a0, 16
-; RV32-NEXT: addi a2, sp, 8
+; RV32-NEXT: mv a2, sp
; RV32-NEXT: li a3, 5
; RV32-NEXT: call __atomic_store
; RV32-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -4667,25 +4667,25 @@ define i128 @cmpxchg128(ptr %p) nounwind {
; RV32-NEXT: sw ra, 44(sp) # 4-byte Folded Spill
; RV32-NEXT: sw s0, 40(sp) # 4-byte Folded Spill
; RV32-NEXT: mv s0, a0
+; RV32-NEXT: sw zero, 16(sp)
+; RV32-NEXT: sw zero, 20(sp)
; RV32-NEXT: sw zero, 24(sp)
; RV32-NEXT: sw zero, 28(sp)
-; RV32-NEXT: sw zero, 32(sp)
-; RV32-NEXT: sw zero, 36(sp)
; RV32-NEXT: li a0, 1
-; RV32-NEXT: sw a0, 8(sp)
+; RV32-NEXT: sw a0, 0(sp)
+; RV32-NEXT: sw zero, 4(sp)
+; RV32-NEXT: sw zero, 8(sp)
; RV32-NEXT: sw zero, 12(sp)
-; RV32-NEXT: sw zero, 16(sp)
-; RV32-NEXT: sw zero, 20(sp)
; RV32-NEXT: li a0, 16
-; RV32-NEXT: addi a2, sp, 24
-; RV32-NEXT: addi a3, sp, 8
+; RV32-NEXT: addi a2, sp, 16
+; RV32-NEXT: mv a3, sp
; RV32-NEXT: li a4, 5
; RV32-NEXT: li a5, 5
; RV32-NEXT: call __atomic_compare_exchange
-; RV32-NEXT: lw a0, 24(sp)
-; RV32-NEXT: lw a1, 28(sp)
-; RV32-NEXT: lw a2, 32(sp)
-; RV32-NEXT: lw a3, 36(sp)
+; RV32-NEXT: lw a0, 16(sp)
+; RV32-NEXT: lw a1, 20(sp)
+; RV32-NEXT: lw a2, 24(sp)
+; RV32-NEXT: lw a3, 28(sp)
; RV32-NEXT: sw a0, 0(s0)
; RV32-NEXT: sw a1, 4(s0)
; RV32-NEXT: sw a2, 8(s0)
diff --git a/llvm/test/CodeGen/RISCV/fp128.ll b/llvm/test/CodeGen/RISCV/fp128.ll
index 9beecf0c025c8..0d9871dcf7b15 100644
--- a/llvm/test/CodeGen/RISCV/fp128.ll
+++ b/llvm/test/CodeGen/RISCV/fp128.ll
@@ -25,16 +25,16 @@ define i32 @test_load_and_cmp() nounwind {
; RV32I-NEXT: lw a6, %lo(y+4)(a0)
; RV32I-NEXT: lw a7, %lo(y+8)(a0)
; RV32I-NEXT: lw a0, %lo(y+12)(a0)
-; RV32I-NEXT: sw a1, 8(sp)
-; RV32I-NEXT: sw a6, 12(sp)
-; RV32I-NEXT: sw a7, 16(sp)
-; RV32I-NEXT: sw a0, 20(sp)
-; RV32I-NEXT: addi a0, sp, 24
-; RV32I-NEXT: addi a1, sp, 8
-; RV32I-NEXT: sw a2, 24(sp)
-; RV32I-NEXT: sw a3, 28(sp)
-; RV32I-NEXT: sw a4, 32(sp)
-; RV32I-NEXT: sw a5, 36(sp)
+; RV32I-NEXT: sw a1, 0(sp)
+; RV32I-NEXT: sw a6, 4(sp)
+; RV32I-NEXT: sw a7, 8(sp)
+; RV32I-NEXT: sw a0, 12(sp)
+; RV32I-NEXT: addi a0, sp, 16
+; RV32I-NEXT: mv a1, sp
+; RV32I-NEXT: sw a2, 16(sp)
+; RV32I-NEXT: sw a3, 20(sp)
+; RV32I-NEXT: sw a4, 24(sp)
+; RV32I-NEXT: sw a5, 28(sp)
; RV32I-NEXT: call __netf2
; RV32I-NEXT: snez a0, a0
; RV32I-NEXT: lw ra, 44(sp) # 4-byte Folded Reload
@@ -78,27 +78,27 @@ define i32 @test_add_and_fptosi() nounwind {
; RV32I-NEXT: lw a2, %lo(y+4)(a0)
; RV32I-NEXT: lw a7, %lo(y+8)(a0)
; RV32I-NEXT: lw a0, %lo(y+12)(a0)
-; RV32I-NEXT: sw a1, 24(sp)
-; RV32I-NEXT: sw a2, 28(sp)
-; RV32I-NEXT: sw a7, 32(sp)
-; RV32I-NEXT: sw a0, 36(sp)
-; RV32I-NEXT: addi a0, sp, 56
-; RV32I-NEXT: addi a1, sp, 40
-; RV32I-NEXT: addi a2, sp, 24
-; RV32I-NEXT: sw a3, 40(sp)
-; RV32I-NEXT: sw a4, 44(sp)
-; RV32I-NEXT: sw a5, 48(sp)
-; RV32I-NEXT: sw a6, 52(sp)
+; RV32I-NEXT: sw a1, 16(sp)
+; RV32I-NEXT: sw a2, 20(sp)
+; RV32I-NEXT: sw a7, 24(sp)
+; RV32I-NEXT: sw a0, 28(sp)
+; RV32I-NEXT: addi a0, sp, 48
+; RV32I-NEXT: addi a1, sp, 32
+; RV32I-NEXT: addi a2, sp, 16
+; RV32I-NEXT: sw a3, 32(sp)
+; RV32I-NEXT: sw a4, 36(sp)
+; RV32I-NEXT: sw a5, 40(sp)
+; RV32I-NEXT: sw a6, 44(sp)
; RV32I-NEXT: call __addtf3
-; RV32I-NEXT: lw a1, 56(sp)
-; RV32I-NEXT: lw a2, 60(sp)
-; RV32I-NEXT: lw a3, 64(sp)
-; RV32I-NEXT: lw a4, 68(sp)
-; RV32I-NEXT: addi a0, sp, 8
-; RV32I-NEXT: sw a1, 8(sp)
-; RV32I-NEXT: sw a2, 12(sp)
-; RV32I-NEXT: sw a3, 16(sp)
-; RV32I-NEXT: sw a4, 20(sp)
+; RV32I-NEXT: lw a1, 48(sp)
+; RV32I-NEXT: lw a2, 52(sp)
+; RV32I-NEXT: lw a3, 56(sp)
+; RV32I-NEXT: lw a4, 60(sp)
+; RV32I-NEXT: mv a0, sp
+; RV32I-NEXT: sw a1, 0(sp)
+; RV32I-NEXT: sw a2, 4(sp)
+; RV32I-NEXT: sw a3, 8(sp)
+; RV32I-NEXT: sw a4, 12(sp)
; RV32I-NEXT: call __fixtfsi
; RV32I-NEXT: lw ra, 76(sp) # 4-byte Folded Reload
; RV32I-NEXT: addi sp, sp, 80
@@ -144,22 +144,22 @@ define fp128 @fmaximum(fp128 %x, fp128 %y) {
; RV32I-NEXT: lw t0, 8(a2)
; RV32I-NEXT: lw a2, 12(a2)
; RV32I-NEXT: mv s0, a0
-; RV32I-NEXT: sw a1, 8(sp)
-; RV32I-NEXT: sw a7, 12(sp)
-; RV32I-NEXT: sw t0, 16(sp)
-; RV32I-NEXT: sw a2, 20(sp)
-; RV32I-NEXT: addi a0, sp, 40
-; RV32I-NEXT: addi a1, sp, 24
-; RV32I-NEXT: addi a2, sp, 8
-; RV32I-NEXT: sw a3, 24(sp)
-; RV32I-NEXT: sw a4, 28(sp)
-; RV32I-NEXT: sw a5, 32(sp)
-; RV32I-NEXT: sw a6, 36(sp)
+; RV32I-NEXT: sw a1, 0(sp)
+; RV32I-NEXT: sw a7, 4(sp)
+; RV32I-NEXT: sw t0, 8(sp)
+; RV32I-NEXT: sw a2, 12(sp)
+; RV32I-NEXT: addi a0, sp, 32
+; RV32I-NEXT: addi a1, sp, 16
+; RV32I-NEXT: mv a2, sp
+; RV32I-NEXT: sw a3, 16(sp)
+; RV32I-NEXT: sw a4, 20(sp)
+; RV32I-NEXT: sw a5, 24(sp)
+; RV32I-NEXT: sw a6, 28(sp)
; RV32I-NEXT: call fmaximuml
-; RV32I-NEXT: lw a0, 40(sp)
-; RV32I-NEXT: lw a1, 44(sp)
-; RV32I-NEXT: lw a2, 48(sp)
-; RV32I-NEXT: lw a3, 52(sp)
+; RV32I-NEXT: lw a0, 32(sp)
+; RV32I-NEXT: lw a1, 36(sp)
+; RV32I-NEXT: lw a2, 40(sp)
+; RV32I-NEXT: lw a3, 44(sp)
; RV32I-NEXT: sw a0, 0(s0)
; RV32I-NEXT: sw a1, 4(s0)
; RV32I-NEXT: sw a2, 8(s0)
@@ -206,22 +206,22 @@ define fp128 @fminimum(fp128 %x, fp128 %y) {
; RV32I-NEXT: lw t0, 8(a2)
; RV32I-NEXT: lw a2, 12(a2)
; RV32I-NEXT: mv s0, a0
-; RV32I-NEXT: sw a1, 8(sp)
-; RV32I-NEXT: sw a7, 12(sp)
-; RV32I-NEXT: sw t0, 16(sp)
-; RV32I-NEXT: sw a2, 20(sp)
-; RV32I-NEXT: addi a0, sp, 40
-; RV32I-NEXT: addi a1, sp, 24
-; RV32I-NEXT: addi a2, sp, 8
-; RV32I-NEXT: sw a3, 24(sp)
-; RV32I-NEXT: sw a4, 28(sp)
-; RV32I-NEXT: sw a5, 32(sp)
-; RV32I-NEXT: sw a6, 36(sp)
+; RV32I-NEXT: sw a1, 0(sp)
+; RV32I-NEXT: sw a7, 4(sp)
+; RV32I-NEXT: sw t0, 8(sp)
+; RV32I-NEXT: sw a2, 12(sp)
+; RV32I-NEXT: addi a0, sp, 32
+; RV32I-NEXT: addi a1, sp, 16
+; RV32I-NEXT: mv a2, sp
+; RV32I-NEXT: sw a3, 16(sp)
+; RV32I-NEXT: sw a4, 20(sp)
+; RV32I-NEXT: sw a5, 24(sp)
+; RV32I-NEXT: sw a6, 28(sp)
; RV32I-NEXT: call fminimuml
-; RV32I-NEXT: lw a0, 40(sp)
-; RV32I-NEXT: lw a1, 44(sp)
-; RV32I-NEXT: lw a2, 48(sp)
-; RV32I-NEXT: lw a3, 52(sp)
+; RV32I-NEXT: lw a0, 32(sp)
+; RV32I-NEXT: lw a1, 36(sp)
+; RV32I-NEXT: lw a2, 40(sp)
+; RV32I-NEXT: lw a3, 44(sp)
; RV32I-NEXT: sw a0, 0(s0)
; RV32I-NEXT: sw a1, 4(s0)
; RV32I-NEXT: sw a2, 8(s0)
@@ -261,22 +261,22 @@ define { fp128, fp128 } @modf(fp128 %a) nounwind {
; RV32I-NEXT: lw a5, 8(a1)
; RV32I-NEXT: lw a6, 12(a1)
; RV32I-NEXT: mv s0, a0
-; RV32I-NEXT: addi a0, sp, 24
-; RV32I-NEXT: addi a1, sp, 8
-; RV32I-NEXT: addi a2, sp, 40
-; RV32I-NEXT: sw a3, 8(sp)
-; RV32I-NEXT: sw a4, 12(sp)
-; RV32I-NEXT: sw a5, 16(sp)
-; RV32I-NEXT: sw a6, 20(sp)
+; RV32I-NEXT: addi a0, sp, 16
+; RV32I-NEXT: mv a1, sp
+; RV32I-NEXT: addi a2, sp, 32
+; RV32I-NEXT: sw a3, 0(sp)
+; RV32I-NEXT: sw a4, 4(sp)
+; RV32I-NEXT: sw a5, 8(sp)
+; RV32I-NEXT: sw a6, 12(sp)
; RV32I-NEXT: call modfl
-; RV32I-NEXT: lw a0, 40(sp)
-; RV32I-NEXT: lw a1, 44(sp)
-; RV32I-NEXT: lw a2, 48(sp)
-; RV32I-NEXT: lw a3, 52(sp)
-; RV32I-NEXT: lw a4, 24(sp)
-; RV32I-NEXT: lw a5, 28(sp)
-; RV32I-NEXT: lw a6, 32(sp)
-; RV32I-NEXT: lw a7, 36(sp)
+; RV32I-NEXT: lw a0, 32(sp)
+; RV32I-NEXT: lw a1, 36(sp)
+; RV32I-NEXT: lw a2, 40(sp)
+; RV32I-NEXT: lw a3, 44(sp)
+; RV32I-NEXT: lw a4, 16(sp)
+; RV32I-NEXT: lw a5, 20(sp)
+; RV32I-NEXT: lw a6, 24(sp)
+; RV32I-NEXT: lw a7, 28(sp)
; RV32I-NEXT: sw a0, 16(s0)
; RV32I-NEXT: sw a1, 20(s0)
; RV32I-NEXT: sw a2, 24(s0)
@@ -369,26 +369,26 @@ define i96 @fptosi_fp128_to_i96(fp128 %a) nounwind {
; RV32I-NEXT: sw a2, 4(sp)
; RV32I-NEXT: sw a0, 8(sp)
; RV32I-NEXT: sw a1, 12(sp)
-; RV32I-NEXT: add a5, a6, a5
-; RV32I-NEXT: lw a0, 12(a5)
-; RV32I-NEXT: lw a1, 8(a5)
-; RV32I-NEXT: lw a2, 4(a5)
-; RV32I-NEXT: lw a3, 0(a5)
+; RV32I-NEXT: or a0, a6, a5
+; RV32I-NEXT: lw a1, 12(a0)
+; RV32I-NEXT: lw a2, 8(a0)
+; RV32I-NEXT: lw a3, 4(a0)
+; RV32I-NEXT: lw a0, 0(a0)
; RV32I-NEXT: andi a5, a4, 31
; RV32I-NEXT: xori a5, a5, 31
-; RV32I-NEXT: slli a6, a0, 1
-; RV32I-NEXT: srl a7, a1, a4
+; RV32I-NEXT: slli a6, a1, 1
+; RV32I-NEXT: srl a7, a2, a4
; RV32I-NEXT: sll a6, a6, a5
; RV32I-NEXT: or s4, a7, a6
-; RV32I-NEXT: slli a1, a1, 1
-; RV32I-NEXT: srl a6, a2, a4
-; RV32I-NEXT: sll a1, a1, a5
-; RV32I-NEXT: or s5, a6, a1
; RV32I-NEXT: slli a2, a2, 1
-; RV32I-NEXT: srl a1, a3, a4
+; RV32I-NEXT: srl a6, a3, a4
; RV32I-NEXT: sll a2, a2, a5
-; RV32I-NEXT: or s6, a1, a2
-; RV32I-NEXT: srl s7, a0, a4
+; RV32I-NEXT: or s5, a6, a2
+; RV32I-NEXT: slli a3, a3, 1
+; RV32I-NEXT: srl a0, a0, a4
+; RV32I-NEXT: sll a2, a3, a5
+; RV32I-NEXT: or s6, a0, a2
+; RV32I-NEXT: srl s7, a1, a4
; RV32I-NEXT: mv a0, s6
; RV32I-NEXT: li a1, 0
; RV32I-NEXT: mv a2, s2
@@ -548,11 +548,11 @@ define i32 @lround_f128() nounwind {
; RV32I-NEXT: lw a2, %lo(x+4)(a0)
; RV32I-NEXT: lw a3, %lo(x+8)(a0)
; RV32I-NEXT: lw a4, %lo(x+12)(a0)
-; RV32I-NEXT: addi a0, sp, 8
-; RV32I-NEXT: sw a1, 8(sp)
-; RV32I-NEXT: sw a2, 12(sp)
-; RV32I-NEXT: sw a3, 16(sp)
-; RV32I-NEXT: sw a4, 20(sp)
+; RV32I-NEXT: mv a0, sp
+; RV32I-NEXT: sw a1, 0(sp)
+; RV32I-NEXT: sw a2, 4(sp)
+; RV32I-NEXT: sw a3, 8(sp)
+; RV32I-NEXT: sw a4, 12(sp)
; RV32I-NEXT: call lroundl
; RV32I-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; RV32I-NEXT: addi sp, sp, 32
diff --git a/llvm/test/CodeGen/RISCV/fpclamptosat.ll b/llvm/test/CodeGen/RISCV/fpclamptosat.ll
index 47b258b6fdb70..1c31b8b0172b3 100644
--- a/llvm/test/CodeGen/RISCV/fpclamptosat.ll
+++ b/llvm/test/CodeGen/RISCV/fpclamptosat.ll
@@ -2580,16 +2580,16 @@ define i64 @stest_f32i64(float %x) {
; RV32-NEXT: .LBB21_4: # %fp-to-i-if-exp.large
; RV32-NEXT: addi a0, a0, -150
; RV32-NEXT: srli a2, a0, 3
-; RV32-NEXT: sw a1, 24(sp)
+; RV32-NEXT: sw a1, 16(sp)
+; RV32-NEXT: sw zero, 20(sp)
+; RV32-NEXT: sw zero, 24(sp)
; RV32-NEXT: sw zero, 28(sp)
-; RV32-NEXT: sw zero, 32(sp)
-; RV32-NEXT: sw zero, 36(sp)
; RV32-NEXT: andi a2, a2, 12
-; RV32-NEXT: addi a1, sp, 24
+; RV32-NEXT: addi a1, sp, 16
+; RV32-NEXT: sw zero, 0(sp)
+; RV32-NEXT: sw zero, 4(sp)
; RV32-NEXT: sw zero, 8(sp)
; RV32-NEXT: sw zero, 12(sp)
-; RV32-NEXT: sw zero, 16(sp)
-; RV32-NEXT: sw zero, 20(sp)
; RV32-NEXT: sub a1, a1, a2
; RV32-NEXT: lw a4, 0(a1)
; RV32-NEXT: lw a2, 4(a1)
@@ -2954,16 +2954,16 @@ define i64 @ustest_f32i64(float %x) {
; RV32-NEXT: .LBB23_5: # %fp-to-i-if-exp.large
; RV32-NEXT: addi a0, a0, -150
; RV32-NEXT: srli a2, a0, 3
-; RV32-NEXT: sw a1, 24(sp)
+; RV32-NEXT: sw a1, 16(sp)
+; RV32-NEXT: sw zero, 20(sp)
+; RV32-NEXT: sw zero, 24(sp)
; RV32-NEXT: sw zero, 28(sp)
-; RV32-NEXT: sw zero, 32(sp)
-; RV32-NEXT: sw zero, 36(sp)
; RV32-NEXT: andi a2, a2, 12
-; RV32-NEXT: addi a1, sp, 24
+; RV32-NEXT: addi a1, sp, 16
+; RV32-NEXT: sw zero, 0(sp)
+; RV32-NEXT: sw zero, 4(sp)
; RV32-NEXT: sw zero, 8(sp)
; RV32-NEXT: sw zero, 12(sp)
-; RV32-NEXT: sw zero, 16(sp)
-; RV32-NEXT: sw zero, 20(sp)
; RV32-NEXT: sub a1, a1, a2
; RV32-NEXT: lw a4, 0(a1)
; RV32-NEXT: lw a2, 4(a1)
@@ -5851,16 +5851,16 @@ define i64 @stest_f32i64_mm(float %x) {
; RV32-NEXT: .LBB48_4: # %fp-to-i-if-exp.large
; RV32-NEXT: addi a0, a0, -150
; RV32-NEXT: srli a2, a0, 3
-; RV32-NEXT: sw a1, 24(sp)
+; RV32-NEXT: sw a1, 16(sp)
+; RV32-NEXT: sw zero, 20(sp)
+; RV32-NEXT: sw zero, 24(sp)
; RV32-NEXT: sw zero, 28(sp)
-; RV32-NEXT: sw zero, 32(sp)
-; RV32-NEXT: sw zero, 36(sp)
; RV32-NEXT: andi a2, a2, 12
-; RV32-NEXT: addi a1, sp, 24
+; RV32-NEXT: addi a1, sp, 16
+; RV32-NEXT: sw zero, 0(sp)
+; RV32-NEXT: sw zero, 4(sp)
; RV32-NEXT: sw zero, 8(sp)
; RV32-NEXT: sw zero, 12(sp)
-; RV32-NEXT: sw zero, 16(sp)
-; RV32-NEXT: sw zero, 20(sp)
; RV32-NEXT: sub a1, a1, a2
; RV32-NEXT: lw a4, 0(a1)
; RV32-NEXT: lw a2, 4(a1)
@@ -6222,16 +6222,16 @@ define i64 @ustest_f32i64_mm(float %x) {
; RV32-NEXT: .LBB50_5: # %fp-to-i-if-exp.large
; RV32-NEXT: addi a0, a0, -150
; RV32-NEXT: srli a2, a0, 3
-; RV32-NEXT: sw a1, 24(sp)
+; RV32-NEXT: sw a1, 16(sp)
+; RV32-NEXT: sw zero, 20(sp)
+; RV32-NEXT: sw zero, 24(sp)
; RV32-NEXT: sw zero, 28(sp)
-; RV32-NEXT: sw zero, 32(sp)
-; RV32-NEXT: sw zero, 36(sp)
; RV32-NEXT: andi a2, a2, 12
-; RV32-NEXT: addi a1, sp, 24
+; RV32-NEXT: addi a1, sp, 16
+; RV32-NEXT: sw zero, 0(sp)
+; RV32-NEXT: sw zero, 4(sp)
; RV32-NEXT: sw zero, 8(sp)
; RV32-NEXT: sw zero, 12(sp)
-; RV32-NEXT: sw zero, 16(sp)
-; RV32-NEXT: sw zero, 20(sp)
; RV32-NEXT: sub a1, a1, a2
; RV32-NEXT: lw a4, 0(a1)
; RV32-NEXT: lw a2, 4(a1)
diff --git a/llvm/test/CodeGen/RISCV/half-convert-strict.ll b/llvm/test/CodeGen/RISCV/half-convert-strict.ll
index 4a6f8a0d9a6d3..98c95d5fee8a5 100644
--- a/llvm/test/CodeGen/RISCV/half-convert-strict.ll
+++ b/llvm/test/CodeGen/RISCV/half-convert-strict.ll
@@ -1790,11 +1790,11 @@ define half @fcvt_h_q(fp128 %a) nounwind strictfp {
; CHECK32-IZFH-NEXT: lw a2, 4(a0)
; CHECK32-IZFH-NEXT: lw a3, 8(a0)
; CHECK32-IZFH-NEXT: lw a4, 12(a0)
-; CHECK32-IZFH-NEXT: addi a0, sp, 8
-; CHECK32-IZFH-NEXT: sw a1, 8(sp)
-; CHECK32-IZFH-NEXT: sw a2, 12(sp)
-; CHECK32-IZFH-NEXT: sw a3, 16(sp)
-; CHECK32-IZFH-NEXT: sw a4, 20(sp)
+; CHECK32-IZFH-NEXT: mv a0, sp
+; CHECK32-IZFH-NEXT: sw a1, 0(sp)
+; CHECK32-IZFH-NEXT: sw a2, 4(sp)
+; CHECK32-IZFH-NEXT: sw a3, 8(sp)
+; CHECK32-IZFH-NEXT: sw a4, 12(sp)
; CHECK32-IZFH-NEXT: call __trunctfhf2
; CHECK32-IZFH-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; CHECK32-IZFH-NEXT: addi sp, sp, 32
@@ -1817,11 +1817,11 @@ define half @fcvt_h_q(fp128 %a) nounwind strictfp {
; CHECK32-IZHINX-NEXT: lw a2, 4(a0)
; CHECK32-IZHINX-NEXT: lw a3, 8(a0)
; CHECK32-IZHINX-NEXT: lw a4, 12(a0)
-; CHECK32-IZHINX-NEXT: addi a0, sp, 8
-; CHECK32-IZHINX-NEXT: sw a1, 8(sp)
-; CHECK32-IZHINX-NEXT: sw a2, 12(sp)
-; CHECK32-IZHINX-NEXT: sw a3, 16(sp)
-; CHECK32-IZHINX-NEXT: sw a4, 20(sp)
+; CHECK32-IZHINX-NEXT: mv a0, sp
+; CHECK32-IZHINX-NEXT: sw a1, 0(sp)
+; CHECK32-IZHINX-NEXT: sw a2, 4(sp)
+; CHECK32-IZHINX-NEXT: sw a3, 8(sp)
+; CHECK32-IZHINX-NEXT: sw a4, 12(sp)
; CHECK32-IZHINX-NEXT: call __trunctfhf2
; CHECK32-IZHINX-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; CHECK32-IZHINX-NEXT: addi sp, sp, 32
@@ -1844,11 +1844,11 @@ define half @fcvt_h_q(fp128 %a) nounwind strictfp {
; CHECK32-IZFHMIN-NEXT: lw a2, 4(a0)
; CHECK32-IZFHMIN-NEXT: lw a3, 8(a0)
; CHECK32-IZFHMIN-NEXT: lw a4, 12(a0)
-; CHECK32-IZFHMIN-NEXT: addi a0, sp, 8
-; CHECK32-IZFHMIN-NEXT: sw a1, 8(sp)
-; CHECK32-IZFHMIN-NEXT: sw a2, 12(sp)
-; CHECK32-IZFHMIN-NEXT: sw a3, 16(sp)
-; CHECK32-IZFHMIN-NEXT: sw a4, 20(sp)
+; CHECK32-IZFHMIN-NEXT: mv a0, sp
+; CHECK32-IZFHMIN-NEXT: sw a1, 0(sp)
+; CHECK32-IZFHMIN-NEXT: sw a2, 4(sp)
+; CHECK32-IZFHMIN-NEXT: sw a3, 8(sp)
+; CHECK32-IZFHMIN-NEXT: sw a4, 12(sp)
; CHECK32-IZFHMIN-NEXT: call __trunctfhf2
; CHECK32-IZFHMIN-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; CHECK32-IZFHMIN-NEXT: addi sp, sp, 32
@@ -1871,11 +1871,11 @@ define half @fcvt_h_q(fp128 %a) nounwind strictfp {
; CHECK32-IZHINXMIN-NEXT: lw a2, 4(a0)
; CHECK32-IZHINXMIN-NEXT: lw a3, 8(a0)
; CHECK32-IZHINXMIN-NEXT: lw a4, 12(a0)
-; CHECK32-IZHINXMIN-NEXT: addi a0, sp, 8
-; CHECK32-IZHINXMIN-NEXT: sw a1, 8(sp)
-; CHECK32-IZHINXMIN-NEXT: sw a2, 12(sp)
-; CHECK32-IZHINXMIN-NEXT: sw a3, 16(sp)
-; CHECK32-IZHINXMIN-NEXT: sw a4, 20(sp)
+; CHECK32-IZHINXMIN-NEXT: mv a0, sp
+; CHECK32-IZHINXMIN-NEXT: sw a1, 0(sp)
+; CHECK32-IZHINXMIN-NEXT: sw a2, 4(sp)
+; CHECK32-IZHINXMIN-NEXT: sw a3, 8(sp)
+; CHECK32-IZHINXMIN-NEXT: sw a4, 12(sp)
; CHECK32-IZHINXMIN-NEXT: call __trunctfhf2
; CHECK32-IZHINXMIN-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; CHECK32-IZHINXMIN-NEXT: addi sp, sp, 32
@@ -1898,11 +1898,11 @@ define half @fcvt_h_q(fp128 %a) nounwind strictfp {
; CHECK32-D-NEXT: lw a2, 4(a0)
; CHECK32-D-NEXT: lw a3, 8(a0)
; CHECK32-D-NEXT: lw a4, 12(a0)
-; CHECK32-D-NEXT: addi a0, sp, 8
-; CHECK32-D-NEXT: sw a1, 8(sp)
-; CHECK32-D-NEXT: sw a2, 12(sp)
-; CHECK32-D-NEXT: sw a3, 16(sp)
-; CHECK32-D-NEXT: sw a4, 20(sp)
+; CHECK32-D-NEXT: mv a0, sp
+; CHECK32-D-NEXT: sw a1, 0(sp)
+; CHECK32-D-NEXT: sw a2, 4(sp)
+; CHECK32-D-NEXT: sw a3, 8(sp)
+; CHECK32-D-NEXT: sw a4, 12(sp)
; CHECK32-D-NEXT: call __trunctfhf2
; CHECK32-D-NEXT: fmv.x.w a0, fa0
; CHECK32-D-NEXT: lui a1, 1048560
@@ -1923,12 +1923,12 @@ define fp128 @fcvt_q_h(half %a) nounwind strictfp {
; CHECK32-IZFH-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; CHECK32-IZFH-NEXT: fcvt.s.h fa0, fa0
; CHECK32-IZFH-NEXT: mv s0, a0
-; CHECK32-IZFH-NEXT: addi a0, sp, 8
+; CHECK32-IZFH-NEXT: mv a0, sp
; CHECK32-IZFH-NEXT: call __extendsftf2
-; CHECK32-IZFH-NEXT: lw a0, 8(sp)
-; CHECK32-IZFH-NEXT: lw a1, 12(sp)
-; CHECK32-IZFH-NEXT: lw a2, 16(sp)
-; CHECK32-IZFH-NEXT: lw a3, 20(sp)
+; CHECK32-IZFH-NEXT: lw a0, 0(sp)
+; CHECK32-IZFH-NEXT: lw a1, 4(sp)
+; CHECK32-IZFH-NEXT: lw a2, 8(sp)
+; CHECK32-IZFH-NEXT: lw a3, 12(sp)
; CHECK32-IZFH-NEXT: sw a0, 0(s0)
; CHECK32-IZFH-NEXT: sw a1, 4(s0)
; CHECK32-IZFH-NEXT: sw a2, 8(s0)
@@ -1955,12 +1955,12 @@ define fp128 @fcvt_q_h(half %a) nounwind strictfp {
; CHECK32-IZHINX-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; CHECK32-IZHINX-NEXT: fcvt.s.h a1, a1
; CHECK32-IZHINX-NEXT: mv s0, a0
-; CHECK32-IZHINX-NEXT: addi a0, sp, 8
+; CHECK32-IZHINX-NEXT: mv a0, sp
; CHECK32-IZHINX-NEXT: call __extendsftf2
-; CHECK32-IZHINX-NEXT: lw a0, 8(sp)
-; CHECK32-IZHINX-NEXT: lw a1, 12(sp)
-; CHECK32-IZHINX-NEXT: lw a2, 16(sp)
-; CHECK32-IZHINX-NEXT: lw a3, 20(sp)
+; CHECK32-IZHINX-NEXT: lw a0, 0(sp)
+; CHECK32-IZHINX-NEXT: lw a1, 4(sp)
+; CHECK32-IZHINX-NEXT: lw a2, 8(sp)
+; CHECK32-IZHINX-NEXT: lw a3, 12(sp)
; CHECK32-IZHINX-NEXT: sw a0, 0(s0)
; CHECK32-IZHINX-NEXT: sw a1, 4(s0)
; CHECK32-IZHINX-NEXT: sw a2, 8(s0)
@@ -1987,12 +1987,12 @@ define fp128 @fcvt_q_h(half %a) nounwind strictfp {
; CHECK32-IZFHMIN-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; CHECK32-IZFHMIN-NEXT: fcvt.s.h fa0, fa0
; CHECK32-IZFHMIN-NEXT: mv s0, a0
-; CHECK32-IZFHMIN-NEXT: addi a0, sp, 8
+; CHECK32-IZFHMIN-NEXT: mv a0, sp
; CHECK32-IZFHMIN-NEXT: call __extendsftf2
-; CHECK32-IZFHMIN-NEXT: lw a0, 8(sp)
-; CHECK32-IZFHMIN-NEXT: lw a1, 12(sp)
-; CHECK32-IZFHMIN-NEXT: lw a2, 16(sp)
-; CHECK32-IZFHMIN-NEXT: lw a3, 20(sp)
+; CHECK32-IZFHMIN-NEXT: lw a0, 0(sp)
+; CHECK32-IZFHMIN-NEXT: lw a1, 4(sp)
+; CHECK32-IZFHMIN-NEXT: lw a2, 8(sp)
+; CHECK32-IZFHMIN-NEXT: lw a3, 12(sp)
; CHECK32-IZFHMIN-NEXT: sw a0, 0(s0)
; CHECK32-IZFHMIN-NEXT: sw a1, 4(s0)
; CHECK32-IZFHMIN-NEXT: sw a2, 8(s0)
@@ -2019,12 +2019,12 @@ define fp128 @fcvt_q_h(half %a) nounwind strictfp {
; CHECK32-IZHINXMIN-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; CHECK32-IZHINXMIN-NEXT: fcvt.s.h a1, a1
; CHECK32-IZHINXMIN-NEXT: mv s0, a0
-; CHECK32-IZHINXMIN-NEXT: addi a0, sp, 8
+; CHECK32-IZHINXMIN-NEXT: mv a0, sp
; CHECK32-IZHINXMIN-NEXT: call __extendsftf2
-; CHECK32-IZHINXMIN-NEXT: lw a0, 8(sp)
-; CHECK32-IZHINXMIN-NEXT: lw a1, 12(sp)
-; CHECK32-IZHINXMIN-NEXT: lw a2, 16(sp)
-; CHECK32-IZHINXMIN-NEXT: lw a3, 20(sp)
+; CHECK32-IZHINXMIN-NEXT: lw a0, 0(sp)
+; CHECK32-IZHINXMIN-NEXT: lw a1, 4(sp)
+; CHECK32-IZHINXMIN-NEXT: lw a2, 8(sp)
+; CHECK32-IZHINXMIN-NEXT: lw a3, 12(sp)
; CHECK32-IZHINXMIN-NEXT: sw a0, 0(s0)
; CHECK32-IZHINXMIN-NEXT: sw a1, 4(s0)
; CHECK32-IZHINXMIN-NEXT: sw a2, 8(s0)
@@ -2055,12 +2055,12 @@ define fp128 @fcvt_q_h(half %a) nounwind strictfp {
; CHECK32-D-NEXT: fmv.w.x fa0, a1
; CHECK32-D-NEXT: mv s0, a0
; CHECK32-D-NEXT: call __extendhfsf2
-; CHECK32-D-NEXT: addi a0, sp, 8
+; CHECK32-D-NEXT: mv a0, sp
; CHECK32-D-NEXT: call __extendsftf2
-; CHECK32-D-NEXT: lw a0, 8(sp)
-; CHECK32-D-NEXT: lw a1, 12(sp)
-; CHECK32-D-NEXT: lw a2, 16(sp)
-; CHECK32-D-NEXT: lw a3, 20(sp)
+; CHECK32-D-NEXT: lw a0, 0(sp)
+; CHECK32-D-NEXT: lw a1, 4(sp)
+; CHECK32-D-NEXT: lw a2, 8(sp)
+; CHECK32-D-NEXT: lw a3, 12(sp)
; CHECK32-D-NEXT: sw a0, 0(s0)
; CHECK32-D-NEXT: sw a1, 4(s0)
; CHECK32-D-NEXT: sw a2, 8(s0)
diff --git a/llvm/test/CodeGen/RISCV/half-convert.ll b/llvm/test/CodeGen/RISCV/half-convert.ll
index 646b13c825195..a16a0e3877868 100644
--- a/llvm/test/CodeGen/RISCV/half-convert.ll
+++ b/llvm/test/CodeGen/RISCV/half-convert.ll
@@ -8668,11 +8668,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; RV32IZFH-NEXT: lw a4, 8(a0)
; RV32IZFH-NEXT: lw a5, 12(a0)
; RV32IZFH-NEXT: mv s0, a1
-; RV32IZFH-NEXT: addi a0, sp, 8
-; RV32IZFH-NEXT: sw a2, 8(sp)
-; RV32IZFH-NEXT: sw a3, 12(sp)
-; RV32IZFH-NEXT: sw a4, 16(sp)
-; RV32IZFH-NEXT: sw a5, 20(sp)
+; RV32IZFH-NEXT: mv a0, sp
+; RV32IZFH-NEXT: sw a2, 0(sp)
+; RV32IZFH-NEXT: sw a3, 4(sp)
+; RV32IZFH-NEXT: sw a4, 8(sp)
+; RV32IZFH-NEXT: sw a5, 12(sp)
; RV32IZFH-NEXT: call __trunctfhf2
; RV32IZFH-NEXT: fsh fa0, 0(s0)
; RV32IZFH-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -8703,11 +8703,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; RV32IDZFH-NEXT: lw a4, 8(a0)
; RV32IDZFH-NEXT: lw a5, 12(a0)
; RV32IDZFH-NEXT: mv s0, a1
-; RV32IDZFH-NEXT: addi a0, sp, 8
-; RV32IDZFH-NEXT: sw a2, 8(sp)
-; RV32IDZFH-NEXT: sw a3, 12(sp)
-; RV32IDZFH-NEXT: sw a4, 16(sp)
-; RV32IDZFH-NEXT: sw a5, 20(sp)
+; RV32IDZFH-NEXT: mv a0, sp
+; RV32IDZFH-NEXT: sw a2, 0(sp)
+; RV32IDZFH-NEXT: sw a3, 4(sp)
+; RV32IDZFH-NEXT: sw a4, 8(sp)
+; RV32IDZFH-NEXT: sw a5, 12(sp)
; RV32IDZFH-NEXT: call __trunctfhf2
; RV32IDZFH-NEXT: fsh fa0, 0(s0)
; RV32IDZFH-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -8738,11 +8738,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; RV32IZHINX-NEXT: lw a4, 8(a0)
; RV32IZHINX-NEXT: lw a5, 12(a0)
; RV32IZHINX-NEXT: mv s0, a1
-; RV32IZHINX-NEXT: addi a0, sp, 8
-; RV32IZHINX-NEXT: sw a2, 8(sp)
-; RV32IZHINX-NEXT: sw a3, 12(sp)
-; RV32IZHINX-NEXT: sw a4, 16(sp)
-; RV32IZHINX-NEXT: sw a5, 20(sp)
+; RV32IZHINX-NEXT: mv a0, sp
+; RV32IZHINX-NEXT: sw a2, 0(sp)
+; RV32IZHINX-NEXT: sw a3, 4(sp)
+; RV32IZHINX-NEXT: sw a4, 8(sp)
+; RV32IZHINX-NEXT: sw a5, 12(sp)
; RV32IZHINX-NEXT: call __trunctfhf2
; RV32IZHINX-NEXT: sh a0, 0(s0)
; RV32IZHINX-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -8773,11 +8773,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; RV32IZDINXZHINX-NEXT: lw a4, 8(a0)
; RV32IZDINXZHINX-NEXT: lw a5, 12(a0)
; RV32IZDINXZHINX-NEXT: mv s0, a1
-; RV32IZDINXZHINX-NEXT: addi a0, sp, 8
-; RV32IZDINXZHINX-NEXT: sw a2, 8(sp)
-; RV32IZDINXZHINX-NEXT: sw a3, 12(sp)
-; RV32IZDINXZHINX-NEXT: sw a4, 16(sp)
-; RV32IZDINXZHINX-NEXT: sw a5, 20(sp)
+; RV32IZDINXZHINX-NEXT: mv a0, sp
+; RV32IZDINXZHINX-NEXT: sw a2, 0(sp)
+; RV32IZDINXZHINX-NEXT: sw a3, 4(sp)
+; RV32IZDINXZHINX-NEXT: sw a4, 8(sp)
+; RV32IZDINXZHINX-NEXT: sw a5, 12(sp)
; RV32IZDINXZHINX-NEXT: call __trunctfhf2
; RV32IZDINXZHINX-NEXT: sh a0, 0(s0)
; RV32IZDINXZHINX-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -8808,11 +8808,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; RV32I-NEXT: lw a4, 8(a0)
; RV32I-NEXT: lw a5, 12(a0)
; RV32I-NEXT: mv s0, a1
-; RV32I-NEXT: addi a0, sp, 8
-; RV32I-NEXT: sw a2, 8(sp)
-; RV32I-NEXT: sw a3, 12(sp)
-; RV32I-NEXT: sw a4, 16(sp)
-; RV32I-NEXT: sw a5, 20(sp)
+; RV32I-NEXT: mv a0, sp
+; RV32I-NEXT: sw a2, 0(sp)
+; RV32I-NEXT: sw a3, 4(sp)
+; RV32I-NEXT: sw a4, 8(sp)
+; RV32I-NEXT: sw a5, 12(sp)
; RV32I-NEXT: call __trunctfhf2
; RV32I-NEXT: sh a0, 0(s0)
; RV32I-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -8843,11 +8843,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; RV32ID-ILP32-NEXT: lw a4, 8(a0)
; RV32ID-ILP32-NEXT: lw a5, 12(a0)
; RV32ID-ILP32-NEXT: mv s0, a1
-; RV32ID-ILP32-NEXT: addi a0, sp, 8
-; RV32ID-ILP32-NEXT: sw a2, 8(sp)
-; RV32ID-ILP32-NEXT: sw a3, 12(sp)
-; RV32ID-ILP32-NEXT: sw a4, 16(sp)
-; RV32ID-ILP32-NEXT: sw a5, 20(sp)
+; RV32ID-ILP32-NEXT: mv a0, sp
+; RV32ID-ILP32-NEXT: sw a2, 0(sp)
+; RV32ID-ILP32-NEXT: sw a3, 4(sp)
+; RV32ID-ILP32-NEXT: sw a4, 8(sp)
+; RV32ID-ILP32-NEXT: sw a5, 12(sp)
; RV32ID-ILP32-NEXT: call __trunctfhf2
; RV32ID-ILP32-NEXT: sh a0, 0(s0)
; RV32ID-ILP32-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -8878,11 +8878,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; RV32ID-NEXT: lw a4, 8(a0)
; RV32ID-NEXT: lw a5, 12(a0)
; RV32ID-NEXT: mv s0, a1
-; RV32ID-NEXT: addi a0, sp, 8
-; RV32ID-NEXT: sw a2, 8(sp)
-; RV32ID-NEXT: sw a3, 12(sp)
-; RV32ID-NEXT: sw a4, 16(sp)
-; RV32ID-NEXT: sw a5, 20(sp)
+; RV32ID-NEXT: mv a0, sp
+; RV32ID-NEXT: sw a2, 0(sp)
+; RV32ID-NEXT: sw a3, 4(sp)
+; RV32ID-NEXT: sw a4, 8(sp)
+; RV32ID-NEXT: sw a5, 12(sp)
; RV32ID-NEXT: call __trunctfhf2
; RV32ID-NEXT: fmv.x.w a0, fa0
; RV32ID-NEXT: sh a0, 0(s0)
@@ -8915,11 +8915,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; CHECK32-IZFHMIN-NEXT: lw a4, 8(a0)
; CHECK32-IZFHMIN-NEXT: lw a5, 12(a0)
; CHECK32-IZFHMIN-NEXT: mv s0, a1
-; CHECK32-IZFHMIN-NEXT: addi a0, sp, 8
-; CHECK32-IZFHMIN-NEXT: sw a2, 8(sp)
-; CHECK32-IZFHMIN-NEXT: sw a3, 12(sp)
-; CHECK32-IZFHMIN-NEXT: sw a4, 16(sp)
-; CHECK32-IZFHMIN-NEXT: sw a5, 20(sp)
+; CHECK32-IZFHMIN-NEXT: mv a0, sp
+; CHECK32-IZFHMIN-NEXT: sw a2, 0(sp)
+; CHECK32-IZFHMIN-NEXT: sw a3, 4(sp)
+; CHECK32-IZFHMIN-NEXT: sw a4, 8(sp)
+; CHECK32-IZFHMIN-NEXT: sw a5, 12(sp)
; CHECK32-IZFHMIN-NEXT: call __trunctfhf2
; CHECK32-IZFHMIN-NEXT: fsh fa0, 0(s0)
; CHECK32-IZFHMIN-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -8950,11 +8950,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; CHECK32-IZHINXMIN-NEXT: lw a4, 8(a0)
; CHECK32-IZHINXMIN-NEXT: lw a5, 12(a0)
; CHECK32-IZHINXMIN-NEXT: mv s0, a1
-; CHECK32-IZHINXMIN-NEXT: addi a0, sp, 8
-; CHECK32-IZHINXMIN-NEXT: sw a2, 8(sp)
-; CHECK32-IZHINXMIN-NEXT: sw a3, 12(sp)
-; CHECK32-IZHINXMIN-NEXT: sw a4, 16(sp)
-; CHECK32-IZHINXMIN-NEXT: sw a5, 20(sp)
+; CHECK32-IZHINXMIN-NEXT: mv a0, sp
+; CHECK32-IZHINXMIN-NEXT: sw a2, 0(sp)
+; CHECK32-IZHINXMIN-NEXT: sw a3, 4(sp)
+; CHECK32-IZHINXMIN-NEXT: sw a4, 8(sp)
+; CHECK32-IZHINXMIN-NEXT: sw a5, 12(sp)
; CHECK32-IZHINXMIN-NEXT: call __trunctfhf2
; CHECK32-IZHINXMIN-NEXT: sh a0, 0(s0)
; CHECK32-IZHINXMIN-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -8985,11 +8985,11 @@ define void @fcvt_h_q(fp128 %x, ptr %y) nounwind {
; CHECK32-IZDINXZHINXMIN-NEXT: lw a4, 8(a0)
; CHECK32-IZDINXZHINXMIN-NEXT: lw a5, 12(a0)
; CHECK32-IZDINXZHINXMIN-NEXT: mv s0, a1
-; CHECK32-IZDINXZHINXMIN-NEXT: addi a0, sp, 8
-; CHECK32-IZDINXZHINXMIN-NEXT: sw a2, 8(sp)
-; CHECK32-IZDINXZHINXMIN-NEXT: sw a3, 12(sp)
-; CHECK32-IZDINXZHINXMIN-NEXT: sw a4, 16(sp)
-; CHECK32-IZDINXZHINXMIN-NEXT: sw a5, 20(sp)
+; CHECK32-IZDINXZHINXMIN-NEXT: mv a0, sp
+; CHECK32-IZDINXZHINXMIN-NEXT: sw a2, 0(sp)
+; CHECK32-IZDINXZHINXMIN-NEXT: sw a3, 4(sp)
+; CHECK32-IZDINXZHINXMIN-NEXT: sw a4, 8(sp)
+; CHECK32-IZDINXZHINXMIN-NEXT: sw a5, 12(sp)
; CHECK32-IZDINXZHINXMIN-NEXT: call __trunctfhf2
; CHECK32-IZDINXZHINXMIN-NEXT: sh a0, 0(s0)
; CHECK32-IZDINXZHINXMIN-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
@@ -9027,12 +9027,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; RV32IZFH-NEXT: mv s0, a0
; RV32IZFH-NEXT: flh fa5, 0(a1)
; RV32IZFH-NEXT: fcvt.s.h fa0, fa5
-; RV32IZFH-NEXT: addi a0, sp, 8
+; RV32IZFH-NEXT: mv a0, sp
; RV32IZFH-NEXT: call __extendsftf2
-; RV32IZFH-NEXT: lw a0, 8(sp)
-; RV32IZFH-NEXT: lw a1, 12(sp)
-; RV32IZFH-NEXT: lw a2, 16(sp)
-; RV32IZFH-NEXT: lw a3, 20(sp)
+; RV32IZFH-NEXT: lw a0, 0(sp)
+; RV32IZFH-NEXT: lw a1, 4(sp)
+; RV32IZFH-NEXT: lw a2, 8(sp)
+; RV32IZFH-NEXT: lw a3, 12(sp)
; RV32IZFH-NEXT: sw a0, 0(s0)
; RV32IZFH-NEXT: sw a1, 4(s0)
; RV32IZFH-NEXT: sw a2, 8(s0)
@@ -9071,12 +9071,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; RV32IDZFH-NEXT: mv s0, a0
; RV32IDZFH-NEXT: flh fa5, 0(a1)
; RV32IDZFH-NEXT: fcvt.s.h fa0, fa5
-; RV32IDZFH-NEXT: addi a0, sp, 8
+; RV32IDZFH-NEXT: mv a0, sp
; RV32IDZFH-NEXT: call __extendsftf2
-; RV32IDZFH-NEXT: lw a0, 8(sp)
-; RV32IDZFH-NEXT: lw a1, 12(sp)
-; RV32IDZFH-NEXT: lw a2, 16(sp)
-; RV32IDZFH-NEXT: lw a3, 20(sp)
+; RV32IDZFH-NEXT: lw a0, 0(sp)
+; RV32IDZFH-NEXT: lw a1, 4(sp)
+; RV32IDZFH-NEXT: lw a2, 8(sp)
+; RV32IDZFH-NEXT: lw a3, 12(sp)
; RV32IDZFH-NEXT: sw a0, 0(s0)
; RV32IDZFH-NEXT: sw a1, 4(s0)
; RV32IDZFH-NEXT: sw a2, 8(s0)
@@ -9115,12 +9115,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; RV32IZHINX-NEXT: mv s0, a0
; RV32IZHINX-NEXT: lh a0, 0(a1)
; RV32IZHINX-NEXT: fcvt.s.h a1, a0
-; RV32IZHINX-NEXT: addi a0, sp, 8
+; RV32IZHINX-NEXT: mv a0, sp
; RV32IZHINX-NEXT: call __extendsftf2
-; RV32IZHINX-NEXT: lw a0, 8(sp)
-; RV32IZHINX-NEXT: lw a1, 12(sp)
-; RV32IZHINX-NEXT: lw a2, 16(sp)
-; RV32IZHINX-NEXT: lw a3, 20(sp)
+; RV32IZHINX-NEXT: lw a0, 0(sp)
+; RV32IZHINX-NEXT: lw a1, 4(sp)
+; RV32IZHINX-NEXT: lw a2, 8(sp)
+; RV32IZHINX-NEXT: lw a3, 12(sp)
; RV32IZHINX-NEXT: sw a0, 0(s0)
; RV32IZHINX-NEXT: sw a1, 4(s0)
; RV32IZHINX-NEXT: sw a2, 8(s0)
@@ -9159,12 +9159,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; RV32IZDINXZHINX-NEXT: mv s0, a0
; RV32IZDINXZHINX-NEXT: lh a0, 0(a1)
; RV32IZDINXZHINX-NEXT: fcvt.s.h a1, a0
-; RV32IZDINXZHINX-NEXT: addi a0, sp, 8
+; RV32IZDINXZHINX-NEXT: mv a0, sp
; RV32IZDINXZHINX-NEXT: call __extendsftf2
-; RV32IZDINXZHINX-NEXT: lw a0, 8(sp)
-; RV32IZDINXZHINX-NEXT: lw a1, 12(sp)
-; RV32IZDINXZHINX-NEXT: lw a2, 16(sp)
-; RV32IZDINXZHINX-NEXT: lw a3, 20(sp)
+; RV32IZDINXZHINX-NEXT: lw a0, 0(sp)
+; RV32IZDINXZHINX-NEXT: lw a1, 4(sp)
+; RV32IZDINXZHINX-NEXT: lw a2, 8(sp)
+; RV32IZDINXZHINX-NEXT: lw a3, 12(sp)
; RV32IZDINXZHINX-NEXT: sw a0, 0(s0)
; RV32IZDINXZHINX-NEXT: sw a1, 4(s0)
; RV32IZDINXZHINX-NEXT: sw a2, 8(s0)
@@ -9204,12 +9204,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; RV32I-NEXT: lh a0, 0(a1)
; RV32I-NEXT: call __extendhfsf2
; RV32I-NEXT: mv a1, a0
-; RV32I-NEXT: addi a0, sp, 8
+; RV32I-NEXT: mv a0, sp
; RV32I-NEXT: call __extendsftf2
-; RV32I-NEXT: lw a0, 8(sp)
-; RV32I-NEXT: lw a1, 12(sp)
-; RV32I-NEXT: lw a2, 16(sp)
-; RV32I-NEXT: lw a3, 20(sp)
+; RV32I-NEXT: lw a0, 0(sp)
+; RV32I-NEXT: lw a1, 4(sp)
+; RV32I-NEXT: lw a2, 8(sp)
+; RV32I-NEXT: lw a3, 12(sp)
; RV32I-NEXT: sw a0, 0(s0)
; RV32I-NEXT: sw a1, 4(s0)
; RV32I-NEXT: sw a2, 8(s0)
@@ -9249,12 +9249,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; RV32ID-ILP32-NEXT: lhu a0, 0(a1)
; RV32ID-ILP32-NEXT: call __extendhfsf2
; RV32ID-ILP32-NEXT: mv a1, a0
-; RV32ID-ILP32-NEXT: addi a0, sp, 8
+; RV32ID-ILP32-NEXT: mv a0, sp
; RV32ID-ILP32-NEXT: call __extendsftf2
-; RV32ID-ILP32-NEXT: lw a0, 8(sp)
-; RV32ID-ILP32-NEXT: lw a1, 12(sp)
-; RV32ID-ILP32-NEXT: lw a2, 16(sp)
-; RV32ID-ILP32-NEXT: lw a3, 20(sp)
+; RV32ID-ILP32-NEXT: lw a0, 0(sp)
+; RV32ID-ILP32-NEXT: lw a1, 4(sp)
+; RV32ID-ILP32-NEXT: lw a2, 8(sp)
+; RV32ID-ILP32-NEXT: lw a3, 12(sp)
; RV32ID-ILP32-NEXT: sw a0, 0(s0)
; RV32ID-ILP32-NEXT: sw a1, 4(s0)
; RV32ID-ILP32-NEXT: sw a2, 8(s0)
@@ -9294,12 +9294,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; RV32ID-NEXT: lhu a0, 0(a1)
; RV32ID-NEXT: fmv.w.x fa0, a0
; RV32ID-NEXT: call __extendhfsf2
-; RV32ID-NEXT: addi a0, sp, 8
+; RV32ID-NEXT: mv a0, sp
; RV32ID-NEXT: call __extendsftf2
-; RV32ID-NEXT: lw a0, 8(sp)
-; RV32ID-NEXT: lw a1, 12(sp)
-; RV32ID-NEXT: lw a2, 16(sp)
-; RV32ID-NEXT: lw a3, 20(sp)
+; RV32ID-NEXT: lw a0, 0(sp)
+; RV32ID-NEXT: lw a1, 4(sp)
+; RV32ID-NEXT: lw a2, 8(sp)
+; RV32ID-NEXT: lw a3, 12(sp)
; RV32ID-NEXT: sw a0, 0(s0)
; RV32ID-NEXT: sw a1, 4(s0)
; RV32ID-NEXT: sw a2, 8(s0)
@@ -9339,12 +9339,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; CHECK32-IZFHMIN-NEXT: mv s0, a0
; CHECK32-IZFHMIN-NEXT: flh fa5, 0(a1)
; CHECK32-IZFHMIN-NEXT: fcvt.s.h fa0, fa5
-; CHECK32-IZFHMIN-NEXT: addi a0, sp, 8
+; CHECK32-IZFHMIN-NEXT: mv a0, sp
; CHECK32-IZFHMIN-NEXT: call __extendsftf2
-; CHECK32-IZFHMIN-NEXT: lw a0, 8(sp)
-; CHECK32-IZFHMIN-NEXT: lw a1, 12(sp)
-; CHECK32-IZFHMIN-NEXT: lw a2, 16(sp)
-; CHECK32-IZFHMIN-NEXT: lw a3, 20(sp)
+; CHECK32-IZFHMIN-NEXT: lw a0, 0(sp)
+; CHECK32-IZFHMIN-NEXT: lw a1, 4(sp)
+; CHECK32-IZFHMIN-NEXT: lw a2, 8(sp)
+; CHECK32-IZFHMIN-NEXT: lw a3, 12(sp)
; CHECK32-IZFHMIN-NEXT: sw a0, 0(s0)
; CHECK32-IZFHMIN-NEXT: sw a1, 4(s0)
; CHECK32-IZFHMIN-NEXT: sw a2, 8(s0)
@@ -9383,12 +9383,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; CHECK32-IZHINXMIN-NEXT: mv s0, a0
; CHECK32-IZHINXMIN-NEXT: lh a0, 0(a1)
; CHECK32-IZHINXMIN-NEXT: fcvt.s.h a1, a0
-; CHECK32-IZHINXMIN-NEXT: addi a0, sp, 8
+; CHECK32-IZHINXMIN-NEXT: mv a0, sp
; CHECK32-IZHINXMIN-NEXT: call __extendsftf2
-; CHECK32-IZHINXMIN-NEXT: lw a0, 8(sp)
-; CHECK32-IZHINXMIN-NEXT: lw a1, 12(sp)
-; CHECK32-IZHINXMIN-NEXT: lw a2, 16(sp)
-; CHECK32-IZHINXMIN-NEXT: lw a3, 20(sp)
+; CHECK32-IZHINXMIN-NEXT: lw a0, 0(sp)
+; CHECK32-IZHINXMIN-NEXT: lw a1, 4(sp)
+; CHECK32-IZHINXMIN-NEXT: lw a2, 8(sp)
+; CHECK32-IZHINXMIN-NEXT: lw a3, 12(sp)
; CHECK32-IZHINXMIN-NEXT: sw a0, 0(s0)
; CHECK32-IZHINXMIN-NEXT: sw a1, 4(s0)
; CHECK32-IZHINXMIN-NEXT: sw a2, 8(s0)
@@ -9427,12 +9427,12 @@ define fp128 @fcvt_q_h(ptr %x) {
; CHECK32-IZDINXZHINXMIN-NEXT: mv s0, a0
; CHECK32-IZDINXZHINXMIN-NEXT: lh a0, 0(a1)
; CHECK32-IZDINXZHINXMIN-NEXT: fcvt.s.h a1, a0
-; CHECK32-IZDINXZHINXMIN-NEXT: addi a0, sp, 8
+; CHECK32-IZDINXZHINXMIN-NEXT: mv a0, sp
; CHECK32-IZDINXZHINXMIN-NEXT: call __extendsftf2
-; CHECK32-IZDINXZHINXMIN-NEXT: lw a0, 8(sp)
-; CHECK32-IZDINXZHINXMIN-NEXT: lw a1, 12(sp)
-; CHECK32-IZDINXZHINXMIN-NEXT: lw a2, 16(sp)
-; CHECK32-IZDINXZHINXMIN-NEXT: lw a3, 20(sp)
+; CHECK32-IZDINXZHINXMIN-NEXT: lw a0, 0(sp)
+; CHECK32-IZDINXZHINXMIN-NEXT: lw a1, 4(sp)
+; CHECK32-IZDINXZHINXMIN-NEXT: lw a2, 8(sp)
+; CHECK32-IZDINXZHINXMIN-NEXT: lw a3, 12(sp)
; CHECK32-IZDINXZHINXMIN-NEXT: sw a0, 0(s0)
; CHECK32-IZDINXZHINXMIN-NEXT: sw a1, 4(s0)
; CHECK32-IZDINXZHINXMIN-NEXT: sw a2, 8(s0)
diff --git a/llvm/test/CodeGen/RISCV/idiv_large.ll b/llvm/test/CodeGen/RISCV/idiv_large.ll
index 27aefeaff7a91..99b4f651ae0eb 100644
--- a/llvm/test/CodeGen/RISCV/idiv_large.ll
+++ b/llvm/test/CodeGen/RISCV/idiv_large.ll
@@ -343,25 +343,25 @@ define i65 @udiv_i65(i65 %x, i65 %y) nounwind {
; RV32-NEXT: sw a4, 4(sp)
; RV32-NEXT: sw t3, 8(sp)
; RV32-NEXT: sw zero, 12(sp)
-; RV32-NEXT: add s5, s6, s5
-; RV32-NEXT: lw a4, 4(s5)
-; RV32-NEXT: srli a5, s2, 1
-; RV32-NEXT: lw t3, 8(s5)
-; RV32-NEXT: srl a5, a5, s1
+; RV32-NEXT: or a4, s6, s5
+; RV32-NEXT: lw a5, 4(a4)
+; RV32-NEXT: srli t3, s2, 1
+; RV32-NEXT: lw s2, 8(a4)
+; RV32-NEXT: srl t3, t3, s1
; RV32-NEXT: sll s1, s4, s3
-; RV32-NEXT: lw s2, 0(s5)
-; RV32-NEXT: or a5, s1, a5
-; RV32-NEXT: srl s1, a4, a6
+; RV32-NEXT: lw a4, 0(a4)
+; RV32-NEXT: or s1, s1, t3
+; RV32-NEXT: srl t3, a5, a6
; RV32-NEXT: andi s3, a6, 31
-; RV32-NEXT: slli t3, t3, 1
+; RV32-NEXT: slli s2, s2, 1
; RV32-NEXT: xori s3, s3, 31
-; RV32-NEXT: sll t3, t3, s3
-; RV32-NEXT: slli a4, a4, 1
-; RV32-NEXT: or t3, s1, t3
-; RV32-NEXT: sll a4, a4, s3
-; RV32-NEXT: andi s1, a5, 1
-; RV32-NEXT: srl a5, s2, a6
-; RV32-NEXT: or s2, a5, a4
+; RV32-NEXT: sll s2, s2, s3
+; RV32-NEXT: slli a5, a5, 1
+; RV32-NEXT: or t3, t3, s2
+; RV32-NEXT: sll a5, a5, s3
+; RV32-NEXT: andi s1, s1, 1
+; RV32-NEXT: srl a4, a4, a6
+; RV32-NEXT: or s2, a4, a5
; RV32-NEXT: snez a4, t1
; RV32-NEXT: seqz a5, a2
; RV32-NEXT: add a4, a1, a4
@@ -786,14 +786,14 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
; RV32-NEXT: or s1, s1, s2
; RV32-NEXT: beqz s1, .LBB2_32
; RV32-NEXT: # %bb.23: # %udiv-bb1
-; RV32-NEXT: sw a3, 8(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a3, 0(sp) # 4-byte Folded Spill
; RV32-NEXT: addi t1, t5, 1
; RV32-NEXT: seqz a0, t1
; RV32-NEXT: add t2, t2, a0
+; RV32-NEXT: sw zero, 48(sp)
+; RV32-NEXT: sw zero, 52(sp)
; RV32-NEXT: sw zero, 56(sp)
; RV32-NEXT: sw zero, 60(sp)
-; RV32-NEXT: sw zero, 64(sp)
-; RV32-NEXT: sw zero, 68(sp)
; RV32-NEXT: or a0, t1, t2
; RV32-NEXT: li a3, 127
; RV32-NEXT: seqz t4, a0
@@ -803,12 +803,12 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
; RV32-NEXT: sltu a4, t4, t6
; RV32-NEXT: andi a3, a3, 12
; RV32-NEXT: add t3, t3, a4
-; RV32-NEXT: addi a4, sp, 72
+; RV32-NEXT: addi a4, sp, 64
; RV32-NEXT: or t6, t2, t3
-; RV32-NEXT: sw t0, 72(sp)
-; RV32-NEXT: sw a6, 76(sp)
-; RV32-NEXT: sw a1, 80(sp)
-; RV32-NEXT: sw a7, 84(sp)
+; RV32-NEXT: sw t0, 64(sp)
+; RV32-NEXT: sw a6, 68(sp)
+; RV32-NEXT: sw a1, 72(sp)
+; RV32-NEXT: sw a7, 76(sp)
; RV32-NEXT: sub a4, a4, a3
; RV32-NEXT: lw s0, 4(a4)
; RV32-NEXT: lw a3, 8(a4)
@@ -839,17 +839,17 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
; RV32-NEXT: li s4, 0
; RV32-NEXT: li s5, 0
; RV32-NEXT: srli a4, t1, 3
+; RV32-NEXT: sw zero, 32(sp)
+; RV32-NEXT: sw zero, 36(sp)
; RV32-NEXT: sw zero, 40(sp)
; RV32-NEXT: sw zero, 44(sp)
-; RV32-NEXT: sw zero, 48(sp)
-; RV32-NEXT: sw zero, 52(sp)
; RV32-NEXT: andi a4, a4, 12
-; RV32-NEXT: addi s0, sp, 24
-; RV32-NEXT: sw t0, 24(sp)
-; RV32-NEXT: sw a6, 28(sp)
-; RV32-NEXT: sw a1, 32(sp)
-; RV32-NEXT: sw a7, 36(sp)
-; RV32-NEXT: add a4, s0, a4
+; RV32-NEXT: addi s0, sp, 16
+; RV32-NEXT: sw t0, 16(sp)
+; RV32-NEXT: sw a6, 20(sp)
+; RV32-NEXT: sw a1, 24(sp)
+; RV32-NEXT: sw a7, 28(sp)
+; RV32-NEXT: or a4, s0, a4
; RV32-NEXT: lw a1, 12(a4)
; RV32-NEXT: lw a6, 8(a4)
; RV32-NEXT: lw a7, 4(a4)
@@ -871,7 +871,7 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
; RV32-NEXT: srl s7, a1, t1
; RV32-NEXT: seqz a1, ra
; RV32-NEXT: or a4, ra, a5
-; RV32-NEXT: sw a5, 20(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a5, 12(sp) # 4-byte Folded Spill
; RV32-NEXT: sub a1, a5, a1
; RV32-NEXT: seqz a4, a4
; RV32-NEXT: sub a6, s10, a4
@@ -879,9 +879,9 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
; RV32-NEXT: sltu a4, s10, a4
; RV32-NEXT: mv a7, s11
; RV32-NEXT: sub a4, s11, a4
-; RV32-NEXT: sw a4, 16(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a4, 8(sp) # 4-byte Folded Spill
; RV32-NEXT: addi a4, ra, -1
-; RV32-NEXT: sw a4, 12(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a4, 4(sp) # 4-byte Folded Spill
; RV32-NEXT: j .LBB2_26
; RV32-NEXT: .LBB2_25: # %udiv-do-while
; RV32-NEXT: # in Loop: Header=BB2_26 Depth=1
@@ -942,7 +942,7 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
; RV32-NEXT: sltu t0, a1, s10
; RV32-NEXT: j .LBB2_29
; RV32-NEXT: .LBB2_28: # in Loop: Header=BB2_26 Depth=1
-; RV32-NEXT: lw a4, 12(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 4(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu t0, a4, s9
; RV32-NEXT: .LBB2_29: # %udiv-do-while
; RV32-NEXT: # in Loop: Header=BB2_26 Depth=1
@@ -954,7 +954,7 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
; RV32-NEXT: or s6, s11, s1
; RV32-NEXT: sub s1, a6, s7
; RV32-NEXT: sltu s8, a6, s7
-; RV32-NEXT: lw a4, 16(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 8(sp) # 4-byte Folded Reload
; RV32-NEXT: sub s11, a4, s6
; RV32-NEXT: sltu t0, s1, t0
; RV32-NEXT: sub s1, s11, s8
@@ -962,7 +962,7 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
; RV32-NEXT: srai s1, s1, 31
; RV32-NEXT: mv a5, ra
; RV32-NEXT: and s11, s1, ra
-; RV32-NEXT: lw a4, 20(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 12(sp) # 4-byte Folded Reload
; RV32-NEXT: and ra, s1, a4
; RV32-NEXT: sltu t0, s9, s11
; RV32-NEXT: mv s8, t0
@@ -983,7 +983,7 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
; RV32-NEXT: slli t5, t5, 1
; RV32-NEXT: or s0, t6, s0
; RV32-NEXT: or a0, a0, t5
-; RV32-NEXT: lw a3, 8(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 0(sp) # 4-byte Folded Reload
; RV32-NEXT: .LBB2_32: # %udiv-end
; RV32-NEXT: sw a0, 0(a3)
; RV32-NEXT: sw t1, 4(a3)
@@ -1020,20 +1020,20 @@ define i128 @udiv_i128(i128 %x, i128 %y) nounwind {
define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-LABEL: udiv_i129:
; RV32: # %bb.0: # %_udiv-special-cases
-; RV32-NEXT: addi sp, sp, -224
-; RV32-NEXT: sw ra, 220(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s0, 216(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s1, 212(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s2, 208(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s3, 204(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s4, 200(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s5, 196(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s6, 192(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s7, 188(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s8, 184(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s9, 180(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s10, 176(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s11, 172(sp) # 4-byte Folded Spill
+; RV32-NEXT: addi sp, sp, -240
+; RV32-NEXT: sw ra, 236(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s0, 232(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s1, 228(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s2, 224(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s3, 220(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s4, 216(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s5, 212(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s6, 208(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s7, 204(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s8, 200(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s9, 196(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s10, 192(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s11, 188(sp) # 4-byte Folded Spill
; RV32-NEXT: mv s7, a0
; RV32-NEXT: lw a5, 4(a2)
; RV32-NEXT: lw a4, 8(a2)
@@ -1042,17 +1042,17 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: lw t2, 16(a2)
; RV32-NEXT: lw a6, 0(a2)
; RV32-NEXT: lw a2, 12(a2)
-; RV32-NEXT: sw a2, 20(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a4, 28(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a2, 28(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a4, 36(sp) # 4-byte Folded Spill
; RV32-NEXT: slli a2, a4, 31
; RV32-NEXT: srli a4, a5, 1
; RV32-NEXT: addi t5, a0, 1365
; RV32-NEXT: or a0, a4, a2
; RV32-NEXT: addi t4, a3, 819
; RV32-NEXT: lui a2, 61681
-; RV32-NEXT: sw a5, 16(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a5, 24(sp) # 4-byte Folded Spill
; RV32-NEXT: slli a3, a5, 31
-; RV32-NEXT: sw a6, 24(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a6, 32(sp) # 4-byte Folded Spill
; RV32-NEXT: srli a4, a6, 1
; RV32-NEXT: addi t3, a2, -241
; RV32-NEXT: or a6, a4, a3
@@ -1114,13 +1114,13 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: add a2, a2, a3
; RV32-NEXT: srli t0, a2, 24
; RV32-NEXT: .LBB3_3: # %_udiv-special-cases
-; RV32-NEXT: lw a5, 20(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a5, 28(sp) # 4-byte Folded Reload
; RV32-NEXT: srli a2, a5, 1
; RV32-NEXT: slli a4, t2, 31
; RV32-NEXT: slli a5, a5, 31
-; RV32-NEXT: lw a3, 24(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 32(sp) # 4-byte Folded Reload
; RV32-NEXT: slli a3, a3, 31
-; RV32-NEXT: lw a7, 28(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a7, 36(sp) # 4-byte Folded Reload
; RV32-NEXT: srli a7, a7, 1
; RV32-NEXT: li s1, 64
; RV32-NEXT: bnez a3, .LBB3_5
@@ -1426,16 +1426,16 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: .LBB3_28: # %_udiv-special-cases
; RV32-NEXT: andi s11, a1, 1
; RV32-NEXT: andi s4, t2, 1
-; RV32-NEXT: lw a1, 28(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s1, 24(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a1, 36(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s1, 32(sp) # 4-byte Folded Reload
; RV32-NEXT: or s1, s1, a1
; RV32-NEXT: or a1, a7, a6
; RV32-NEXT: bnez s5, .LBB3_30
; RV32-NEXT: # %bb.29: # %_udiv-special-cases
; RV32-NEXT: mv a4, a3
; RV32-NEXT: .LBB3_30: # %_udiv-special-cases
-; RV32-NEXT: lw a2, 16(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw a3, 20(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 24(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 28(sp) # 4-byte Folded Reload
; RV32-NEXT: or s6, a2, a3
; RV32-NEXT: or a5, s1, s4
; RV32-NEXT: or a2, t0, t1
@@ -1494,37 +1494,37 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: or t6, t6, s2
; RV32-NEXT: beqz t6, .LBB3_55
; RV32-NEXT: # %bb.40: # %udiv-bb1
-; RV32-NEXT: sw s4, 36(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s7, 0(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s4, 44(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s7, 8(sp) # 4-byte Folded Spill
; RV32-NEXT: addi a1, a0, 1
; RV32-NEXT: seqz a2, a1
; RV32-NEXT: add t2, t2, a2
; RV32-NEXT: or a2, a1, t2
; RV32-NEXT: seqz t4, a2
-; RV32-NEXT: sw zero, 120(sp)
-; RV32-NEXT: sw zero, 124(sp)
; RV32-NEXT: sw zero, 128(sp)
; RV32-NEXT: sw zero, 132(sp)
+; RV32-NEXT: sw zero, 136(sp)
+; RV32-NEXT: sw zero, 140(sp)
; RV32-NEXT: add t4, a3, t4
; RV32-NEXT: sltu a2, t4, a3
-; RV32-NEXT: sw zero, 104(sp)
-; RV32-NEXT: sw zero, 108(sp)
; RV32-NEXT: sw zero, 112(sp)
; RV32-NEXT: sw zero, 116(sp)
+; RV32-NEXT: sw zero, 120(sp)
+; RV32-NEXT: sw zero, 124(sp)
; RV32-NEXT: add t3, t3, a2
; RV32-NEXT: or a2, a1, t4
; RV32-NEXT: or a3, t2, t3
; RV32-NEXT: li a4, 128
; RV32-NEXT: or a5, a2, a3
; RV32-NEXT: sub a3, a4, a0
-; RV32-NEXT: sw a7, 136(sp)
-; RV32-NEXT: sw t0, 140(sp)
-; RV32-NEXT: sw a6, 144(sp)
-; RV32-NEXT: sw t1, 148(sp)
+; RV32-NEXT: sw a7, 144(sp)
+; RV32-NEXT: sw t0, 148(sp)
+; RV32-NEXT: sw a6, 152(sp)
+; RV32-NEXT: sw t1, 156(sp)
; RV32-NEXT: srli a2, a3, 3
; RV32-NEXT: andi a2, a2, 28
-; RV32-NEXT: addi a4, sp, 136
-; RV32-NEXT: sw s11, 152(sp)
+; RV32-NEXT: addi a4, sp, 144
+; RV32-NEXT: sw s11, 160(sp)
; RV32-NEXT: sub a4, a4, a2
; RV32-NEXT: lw t6, 4(a4)
; RV32-NEXT: lw s0, 8(a4)
@@ -1558,25 +1558,25 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: li s10, 0
; RV32-NEXT: srli s5, a2, 1
; RV32-NEXT: lw ra, 16(a4)
-; RV32-NEXT: sw zero, 88(sp)
-; RV32-NEXT: sw zero, 92(sp)
; RV32-NEXT: sw zero, 96(sp)
; RV32-NEXT: sw zero, 100(sp)
-; RV32-NEXT: sw zero, 72(sp)
-; RV32-NEXT: sw zero, 76(sp)
+; RV32-NEXT: sw zero, 104(sp)
+; RV32-NEXT: sw zero, 108(sp)
; RV32-NEXT: sw zero, 80(sp)
; RV32-NEXT: sw zero, 84(sp)
+; RV32-NEXT: sw zero, 88(sp)
+; RV32-NEXT: sw zero, 92(sp)
; RV32-NEXT: srli a2, a1, 3
-; RV32-NEXT: sw s11, 56(sp)
-; RV32-NEXT: sw zero, 60(sp)
-; RV32-NEXT: sw zero, 64(sp)
+; RV32-NEXT: sw s11, 64(sp)
; RV32-NEXT: sw zero, 68(sp)
+; RV32-NEXT: sw zero, 72(sp)
+; RV32-NEXT: sw zero, 76(sp)
; RV32-NEXT: andi a2, a2, 28
-; RV32-NEXT: addi a4, sp, 40
-; RV32-NEXT: sw a7, 40(sp)
-; RV32-NEXT: sw t0, 44(sp)
-; RV32-NEXT: sw a6, 48(sp)
-; RV32-NEXT: sw t1, 52(sp)
+; RV32-NEXT: addi a4, sp, 48
+; RV32-NEXT: sw a7, 48(sp)
+; RV32-NEXT: sw t0, 52(sp)
+; RV32-NEXT: sw a6, 56(sp)
+; RV32-NEXT: sw t1, 60(sp)
; RV32-NEXT: add a2, a4, a2
; RV32-NEXT: lw a4, 16(a2)
; RV32-NEXT: lw a5, 4(a2)
@@ -1606,34 +1606,34 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: srl a2, a2, a1
; RV32-NEXT: snez a3, a3
; RV32-NEXT: slli a5, a5, 1
-; RV32-NEXT: lw a4, 36(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 44(sp) # 4-byte Folded Reload
; RV32-NEXT: add a3, a4, a3
; RV32-NEXT: sll a4, a5, t0
; RV32-NEXT: not a3, a3
; RV32-NEXT: or s6, a2, a4
; RV32-NEXT: andi a3, a3, 1
-; RV32-NEXT: sw a3, 12(sp) # 4-byte Folded Spill
-; RV32-NEXT: lw a5, 24(sp) # 4-byte Folded Reload
+; RV32-NEXT: sw a3, 20(sp) # 4-byte Folded Spill
+; RV32-NEXT: lw a5, 32(sp) # 4-byte Folded Reload
; RV32-NEXT: seqz a2, a5
-; RV32-NEXT: lw a4, 16(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 24(sp) # 4-byte Folded Reload
; RV32-NEXT: or a3, a5, a4
; RV32-NEXT: sub a6, a4, a2
; RV32-NEXT: seqz a2, a3
-; RV32-NEXT: lw a3, 28(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 36(sp) # 4-byte Folded Reload
; RV32-NEXT: sub a4, a3, a2
-; RV32-NEXT: sw a4, 32(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a4, 40(sp) # 4-byte Folded Spill
; RV32-NEXT: sltu a2, a3, a2
-; RV32-NEXT: lw a3, 20(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a3, 28(sp) # 4-byte Folded Reload
; RV32-NEXT: sub s4, a3, a2
; RV32-NEXT: addi a5, a5, -1
-; RV32-NEXT: sw a5, 4(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw a6, 8(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a5, 12(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw a6, 16(sp) # 4-byte Folded Spill
; RV32-NEXT: mv t1, t5
; RV32-NEXT: j .LBB3_43
; RV32-NEXT: .LBB3_42: # %udiv-do-while
; RV32-NEXT: # in Loop: Header=BB3_43 Depth=1
; RV32-NEXT: mv s8, s7
-; RV32-NEXT: lw a7, 28(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a7, 36(sp) # 4-byte Folded Reload
; RV32-NEXT: and a7, s1, a7
; RV32-NEXT: xor ra, a3, s11
; RV32-NEXT: xor t0, a0, a7
@@ -1665,7 +1665,7 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: sub a3, a3, t0
; RV32-NEXT: sub s11, a3, a5
; RV32-NEXT: sub a2, s5, a2
-; RV32-NEXT: lw a7, 36(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a7, 44(sp) # 4-byte Folded Reload
; RV32-NEXT: or a3, a1, a7
; RV32-NEXT: or a5, t2, t3
; RV32-NEXT: sub s5, a2, a6
@@ -1691,7 +1691,7 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: li s8, 0
; RV32-NEXT: li s9, 0
; RV32-NEXT: li s10, 0
-; RV32-NEXT: lw a6, 8(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a6, 16(sp) # 4-byte Folded Reload
; RV32-NEXT: beqz a2, .LBB3_54
; RV32-NEXT: .LBB3_43: # %udiv-do-while
; RV32-NEXT: # =>This Inner Loop Header: Depth=1
@@ -1707,11 +1707,11 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: sltu a2, s4, a3
; RV32-NEXT: j .LBB3_46
; RV32-NEXT: .LBB3_45: # in Loop: Header=BB3_43 Depth=1
-; RV32-NEXT: lw a2, 32(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 40(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a2, a2, a0
; RV32-NEXT: .LBB3_46: # %udiv-do-while
; RV32-NEXT: # in Loop: Header=BB3_43 Depth=1
-; RV32-NEXT: sw t4, 36(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw t4, 44(sp) # 4-byte Folded Spill
; RV32-NEXT: srli a4, s6, 31
; RV32-NEXT: slli s5, s5, 1
; RV32-NEXT: or s5, s5, a4
@@ -1725,13 +1725,13 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: sltu a4, a6, s5
; RV32-NEXT: j .LBB3_49
; RV32-NEXT: .LBB3_48: # in Loop: Header=BB3_43 Depth=1
-; RV32-NEXT: lw a4, 4(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 12(sp) # 4-byte Folded Reload
; RV32-NEXT: sltu a4, a4, s6
; RV32-NEXT: .LBB3_49: # %udiv-do-while
; RV32-NEXT: # in Loop: Header=BB3_43 Depth=1
; RV32-NEXT: mv s3, s10
; RV32-NEXT: mv t5, s1
-; RV32-NEXT: lw a5, 32(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a5, 40(sp) # 4-byte Folded Reload
; RV32-NEXT: xor a5, a5, a0
; RV32-NEXT: xor a6, s4, a3
; RV32-NEXT: or a5, a5, a6
@@ -1743,17 +1743,17 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: # in Loop: Header=BB3_43 Depth=1
; RV32-NEXT: mv s10, s9
; RV32-NEXT: srli a2, s11, 31
-; RV32-NEXT: lw a5, 12(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a5, 20(sp) # 4-byte Folded Reload
; RV32-NEXT: sub a2, a5, a2
; RV32-NEXT: sub a2, a2, a4
; RV32-NEXT: slli a2, a2, 31
; RV32-NEXT: srai s1, a2, 31
-; RV32-NEXT: lw a2, 20(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 28(sp) # 4-byte Folded Reload
; RV32-NEXT: and s11, s1, a2
; RV32-NEXT: mv s9, s8
-; RV32-NEXT: lw a4, 24(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 32(sp) # 4-byte Folded Reload
; RV32-NEXT: and a4, s1, a4
-; RV32-NEXT: lw a2, 16(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a2, 24(sp) # 4-byte Folded Reload
; RV32-NEXT: and a2, s1, a2
; RV32-NEXT: sltu a6, s6, a4
; RV32-NEXT: mv a5, a6
@@ -1778,27 +1778,27 @@ define i129 @udiv_i129(i129 %x, i129 %y) nounwind {
; RV32-NEXT: slli t6, t6, 1
; RV32-NEXT: srli a5, t1, 31
; RV32-NEXT: or t4, s1, t6
-; RV32-NEXT: lw s7, 0(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s7, 8(sp) # 4-byte Folded Reload
; RV32-NEXT: .LBB3_55: # %udiv-end
; RV32-NEXT: sw t4, 0(s7)
; RV32-NEXT: sw a2, 4(s7)
; RV32-NEXT: sw a1, 8(s7)
; RV32-NEXT: sw a4, 12(s7)
; RV32-NEXT: sb a5, 16(s7)
-; RV32-NEXT: lw ra, 220(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s0, 216(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s1, 212(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s2, 208(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s3, 204(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s4, 200(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s5, 196(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s6, 192(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s7, 188(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s8, 184(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s9, 180(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s10, 176(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s11, 172(sp) # 4-byte Folded Reload
-; RV32-NEXT: addi sp, sp, 224
+; RV32-NEXT: lw ra, 236(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s0, 232(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s1, 228(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s2, 224(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s3, 220(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s4, 216(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s5, 212(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s6, 208(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s7, 204(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s8, 200(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s9, 196(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s10, 192(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s11, 188(sp) # 4-byte Folded Reload
+; RV32-NEXT: addi sp, sp, 240
; RV32-NEXT: ret
;
; RV64-LABEL: udiv_i129:
diff --git a/llvm/test/CodeGen/RISCV/llvm.frexp.ll b/llvm/test/CodeGen/RISCV/llvm.frexp.ll
index aacab9d6ccced..76695b89b54bc 100644
--- a/llvm/test/CodeGen/RISCV/llvm.frexp.ll
+++ b/llvm/test/CodeGen/RISCV/llvm.frexp.ll
@@ -1841,13 +1841,13 @@ define i32 @test_frexp_f128_i32_only_use_exp(fp128 %a) nounwind {
; RV32IFD-NEXT: lw a4, 4(a0)
; RV32IFD-NEXT: lw a5, 8(a0)
; RV32IFD-NEXT: lw a6, 12(a0)
-; RV32IFD-NEXT: addi a0, sp, 24
-; RV32IFD-NEXT: addi a1, sp, 8
+; RV32IFD-NEXT: addi a0, sp, 16
+; RV32IFD-NEXT: mv a1, sp
; RV32IFD-NEXT: addi a2, sp, 40
-; RV32IFD-NEXT: sw a3, 8(sp)
-; RV32IFD-NEXT: sw a4, 12(sp)
-; RV32IFD-NEXT: sw a5, 16(sp)
-; RV32IFD-NEXT: sw a6, 20(sp)
+; RV32IFD-NEXT: sw a3, 0(sp)
+; RV32IFD-NEXT: sw a4, 4(sp)
+; RV32IFD-NEXT: sw a5, 8(sp)
+; RV32IFD-NEXT: sw a6, 12(sp)
; RV32IFD-NEXT: call frexpl
; RV32IFD-NEXT: lw a0, 40(sp)
; RV32IFD-NEXT: lw ra, 44(sp) # 4-byte Folded Reload
@@ -1873,13 +1873,13 @@ define i32 @test_frexp_f128_i32_only_use_exp(fp128 %a) nounwind {
; RV32IZFINXZDINX-NEXT: lw a4, 4(a0)
; RV32IZFINXZDINX-NEXT: lw a5, 8(a0)
; RV32IZFINXZDINX-NEXT: lw a6, 12(a0)
-; RV32IZFINXZDINX-NEXT: addi a0, sp, 24
-; RV32IZFINXZDINX-NEXT: addi a1, sp, 8
+; RV32IZFINXZDINX-NEXT: addi a0, sp, 16
+; RV32IZFINXZDINX-NEXT: mv a1, sp
; RV32IZFINXZDINX-NEXT: addi a2, sp, 40
-; RV32IZFINXZDINX-NEXT: sw a3, 8(sp)
-; RV32IZFINXZDINX-NEXT: sw a4, 12(sp)
-; RV32IZFINXZDINX-NEXT: sw a5, 16(sp)
-; RV32IZFINXZDINX-NEXT: sw a6, 20(sp)
+; RV32IZFINXZDINX-NEXT: sw a3, 0(sp)
+; RV32IZFINXZDINX-NEXT: sw a4, 4(sp)
+; RV32IZFINXZDINX-NEXT: sw a5, 8(sp)
+; RV32IZFINXZDINX-NEXT: sw a6, 12(sp)
; RV32IZFINXZDINX-NEXT: call frexpl
; RV32IZFINXZDINX-NEXT: lw a0, 40(sp)
; RV32IZFINXZDINX-NEXT: lw ra, 44(sp) # 4-byte Folded Reload
@@ -1905,13 +1905,13 @@ define i32 @test_frexp_f128_i32_only_use_exp(fp128 %a) nounwind {
; RV32I-NEXT: lw a4, 4(a0)
; RV32I-NEXT: lw a5, 8(a0)
; RV32I-NEXT: lw a6, 12(a0)
-; RV32I-NEXT: addi a0, sp, 24
-; RV32I-NEXT: addi a1, sp, 8
+; RV32I-NEXT: addi a0, sp, 16
+; RV32I-NEXT: mv a1, sp
; RV32I-NEXT: addi a2, sp, 40
-; RV32I-NEXT: sw a3, 8(sp)
-; RV32I-NEXT: sw a4, 12(sp)
-; RV32I-NEXT: sw a5, 16(sp)
-; RV32I-NEXT: sw a6, 20(sp)
+; RV32I-NEXT: sw a3, 0(sp)
+; RV32I-NEXT: sw a4, 4(sp)
+; RV32I-NEXT: sw a5, 8(sp)
+; RV32I-NEXT: sw a6, 12(sp)
; RV32I-NEXT: call frexpl
; RV32I-NEXT: lw a0, 40(sp)
; RV32I-NEXT: lw ra, 44(sp) # 4-byte Folded Reload
diff --git a/llvm/test/CodeGen/RISCV/musttail-indirect-args.ll b/llvm/test/CodeGen/RISCV/musttail-indirect-args.ll
index d98a0ab3594fa..f650242a2ccb6 100644
--- a/llvm/test/CodeGen/RISCV/musttail-indirect-args.ll
+++ b/llvm/test/CodeGen/RISCV/musttail-indirect-args.ll
@@ -366,22 +366,22 @@ define i32 @caller_musttail_computed(fp128 %a) nounwind {
; RV32-NEXT: lw a3, 4(s0)
; RV32-NEXT: lw a4, 8(s0)
; RV32-NEXT: lw a5, 12(s0)
-; RV32-NEXT: sw a0, 8(sp)
-; RV32-NEXT: sw a0, 24(sp)
-; RV32-NEXT: sw a3, 12(sp)
-; RV32-NEXT: sw a4, 16(sp)
-; RV32-NEXT: sw a5, 20(sp)
-; RV32-NEXT: addi a0, sp, 40
-; RV32-NEXT: addi a1, sp, 24
-; RV32-NEXT: addi a2, sp, 8
-; RV32-NEXT: sw a3, 28(sp)
-; RV32-NEXT: sw a4, 32(sp)
-; RV32-NEXT: sw a5, 36(sp)
+; RV32-NEXT: sw a0, 0(sp)
+; RV32-NEXT: sw a0, 16(sp)
+; RV32-NEXT: sw a3, 4(sp)
+; RV32-NEXT: sw a4, 8(sp)
+; RV32-NEXT: sw a5, 12(sp)
+; RV32-NEXT: addi a0, sp, 32
+; RV32-NEXT: addi a1, sp, 16
+; RV32-NEXT: mv a2, sp
+; RV32-NEXT: sw a3, 20(sp)
+; RV32-NEXT: sw a4, 24(sp)
+; RV32-NEXT: sw a5, 28(sp)
; RV32-NEXT: call __addtf3
-; RV32-NEXT: lw a0, 40(sp)
-; RV32-NEXT: lw a1, 44(sp)
-; RV32-NEXT: lw a2, 48(sp)
-; RV32-NEXT: lw a3, 52(sp)
+; RV32-NEXT: lw a0, 32(sp)
+; RV32-NEXT: lw a1, 36(sp)
+; RV32-NEXT: lw a2, 40(sp)
+; RV32-NEXT: lw a3, 44(sp)
; RV32-NEXT: sw a0, 0(s0)
; RV32-NEXT: sw a1, 4(s0)
; RV32-NEXT: mv a0, s0
@@ -584,20 +584,20 @@ define i32 @caller_musttail_forwarded_and_computed(fp128 %a, fp128 %b) nounwind
define i32 @caller_musttail_both_computed(fp128 %a, fp128 %b) nounwind {
; RV32-LABEL: caller_musttail_both_computed:
; RV32: # %bb.0:
-; RV32-NEXT: addi sp, sp, -160
-; RV32-NEXT: sw ra, 156(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s0, 152(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s1, 148(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s2, 144(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s3, 140(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s4, 136(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s5, 132(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s6, 128(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s7, 124(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s8, 120(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s9, 116(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s10, 112(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s11, 108(sp) # 4-byte Folded Spill
+; RV32-NEXT: addi sp, sp, -176
+; RV32-NEXT: sw ra, 172(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s0, 168(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s1, 164(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s2, 160(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s3, 156(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s4, 152(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s5, 148(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s6, 144(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s7, 140(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s8, 136(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s9, 132(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s10, 128(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s11, 124(sp) # 4-byte Folded Spill
; RV32-NEXT: mv s0, a1
; RV32-NEXT: mv s1, a0
; RV32-NEXT: lw s5, 0(a1)
@@ -608,45 +608,45 @@ define i32 @caller_musttail_both_computed(fp128 %a, fp128 %b) nounwind {
; RV32-NEXT: lw s7, 4(a0)
; RV32-NEXT: lw s8, 8(a0)
; RV32-NEXT: lw s9, 12(a0)
-; RV32-NEXT: sw s6, 72(sp)
-; RV32-NEXT: sw s7, 76(sp)
-; RV32-NEXT: sw s8, 80(sp)
-; RV32-NEXT: sw s9, 84(sp)
-; RV32-NEXT: addi a0, sp, 88
-; RV32-NEXT: addi a1, sp, 72
-; RV32-NEXT: addi a2, sp, 56
-; RV32-NEXT: sw s5, 56(sp)
-; RV32-NEXT: sw s2, 60(sp)
-; RV32-NEXT: sw s3, 64(sp)
-; RV32-NEXT: sw s4, 68(sp)
+; RV32-NEXT: sw s6, 80(sp)
+; RV32-NEXT: sw s7, 84(sp)
+; RV32-NEXT: sw s8, 88(sp)
+; RV32-NEXT: sw s9, 92(sp)
+; RV32-NEXT: addi a0, sp, 96
+; RV32-NEXT: addi a1, sp, 80
+; RV32-NEXT: addi a2, sp, 64
+; RV32-NEXT: sw s5, 64(sp)
+; RV32-NEXT: sw s2, 68(sp)
+; RV32-NEXT: sw s3, 72(sp)
+; RV32-NEXT: sw s4, 76(sp)
; RV32-NEXT: call __addtf3
-; RV32-NEXT: lw s10, 88(sp)
-; RV32-NEXT: lw s11, 92(sp)
-; RV32-NEXT: lw a0, 96(sp)
-; RV32-NEXT: sw a0, 0(sp) # 4-byte Folded Spill
-; RV32-NEXT: lw a0, 100(sp)
-; RV32-NEXT: sw a0, 4(sp) # 4-byte Folded Spill
-; RV32-NEXT: sw s6, 24(sp)
-; RV32-NEXT: sw s7, 28(sp)
-; RV32-NEXT: sw s8, 32(sp)
-; RV32-NEXT: sw s9, 36(sp)
-; RV32-NEXT: addi a0, sp, 40
-; RV32-NEXT: addi a1, sp, 24
-; RV32-NEXT: addi a2, sp, 8
-; RV32-NEXT: sw s5, 8(sp)
-; RV32-NEXT: sw s2, 12(sp)
-; RV32-NEXT: sw s3, 16(sp)
-; RV32-NEXT: sw s4, 20(sp)
+; RV32-NEXT: lw s10, 96(sp)
+; RV32-NEXT: lw s11, 100(sp)
+; RV32-NEXT: lw a0, 104(sp)
+; RV32-NEXT: sw a0, 8(sp) # 4-byte Folded Spill
+; RV32-NEXT: lw a0, 108(sp)
+; RV32-NEXT: sw a0, 12(sp) # 4-byte Folded Spill
+; RV32-NEXT: sw s6, 32(sp)
+; RV32-NEXT: sw s7, 36(sp)
+; RV32-NEXT: sw s8, 40(sp)
+; RV32-NEXT: sw s9, 44(sp)
+; RV32-NEXT: addi a0, sp, 48
+; RV32-NEXT: addi a1, sp, 32
+; RV32-NEXT: addi a2, sp, 16
+; RV32-NEXT: sw s5, 16(sp)
+; RV32-NEXT: sw s2, 20(sp)
+; RV32-NEXT: sw s3, 24(sp)
+; RV32-NEXT: sw s4, 28(sp)
; RV32-NEXT: call __subtf3
-; RV32-NEXT: lw a0, 40(sp)
-; RV32-NEXT: lw a1, 44(sp)
-; RV32-NEXT: lw a2, 48(sp)
-; RV32-NEXT: lw a3, 52(sp)
+; RV32-NEXT: lw a0, 48(sp)
+; RV32-NEXT: lw a1, 52(sp)
+; RV32-NEXT: lw a2, 56(sp)
+; RV32-NEXT: lw a3, 60(sp)
; RV32-NEXT: sw s10, 0(s1)
; RV32-NEXT: sw s11, 4(s1)
-; RV32-NEXT: lw a4, 0(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 8(sp) # 4-byte Folded Reload
; RV32-NEXT: sw a4, 8(s1)
-; RV32-NEXT: lw a4, 4(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw a4, 12(sp) # 4-byte Folded Reload
; RV32-NEXT: sw a4, 12(s1)
; RV32-NEXT: sw a0, 0(s0)
; RV32-NEXT: sw a1, 4(s0)
@@ -654,20 +654,20 @@ define i32 @caller_musttail_both_computed(fp128 %a, fp128 %b) nounwind {
; RV32-NEXT: mv a1, s0
; RV32-NEXT: sw a2, 8(s0)
; RV32-NEXT: sw a3, 12(s0)
-; RV32-NEXT: lw ra, 156(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s0, 152(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s1, 148(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s2, 144(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s3, 140(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s4, 136(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s5, 132(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s6, 128(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s7, 124(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s8, 120(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s9, 116(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s10, 112(sp) # 4-byte Folded Reload
-; RV32-NEXT: lw s11, 108(sp) # 4-byte Folded Reload
-; RV32-NEXT: addi sp, sp, 160
+; RV32-NEXT: lw ra, 172(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s0, 168(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s1, 164(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s2, 160(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s3, 156(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s4, 152(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s5, 148(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s6, 144(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s7, 140(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s8, 136(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s9, 132(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s10, 128(sp) # 4-byte Folded Reload
+; RV32-NEXT: lw s11, 124(sp) # 4-byte Folded Reload
+; RV32-NEXT: addi sp, sp, 176
; RV32-NEXT: tail callee_musttail_two_indirect
;
; RV64-LABEL: caller_musttail_both_computed:
diff --git a/llvm/test/CodeGen/RISCV/shifts.ll b/llvm/test/CodeGen/RISCV/shifts.ll
index fd95db5bfb99a..15fc9b28c2946 100644
--- a/llvm/test/CodeGen/RISCV/shifts.ll
+++ b/llvm/test/CodeGen/RISCV/shifts.ll
@@ -165,30 +165,30 @@ define i128 @lshr128(i128 %a, i128 %b) nounwind {
; RV32I-NEXT: sw a4, 4(sp)
; RV32I-NEXT: sw a5, 8(sp)
; RV32I-NEXT: sw a1, 12(sp)
-; RV32I-NEXT: add a6, a7, a6
-; RV32I-NEXT: lw a1, 4(a6)
-; RV32I-NEXT: lw a3, 0(a6)
-; RV32I-NEXT: lw a4, 8(a6)
-; RV32I-NEXT: andi a5, a2, 31
-; RV32I-NEXT: lw a6, 12(a6)
-; RV32I-NEXT: slli a7, a1, 1
-; RV32I-NEXT: xori a5, a5, 31
+; RV32I-NEXT: or a1, a7, a6
+; RV32I-NEXT: lw a3, 4(a1)
+; RV32I-NEXT: lw a4, 0(a1)
+; RV32I-NEXT: lw a5, 8(a1)
+; RV32I-NEXT: andi a6, a2, 31
+; RV32I-NEXT: lw a1, 12(a1)
+; RV32I-NEXT: slli a7, a3, 1
+; RV32I-NEXT: xori a6, a6, 31
+; RV32I-NEXT: srl a4, a4, a2
+; RV32I-NEXT: sll a7, a7, a6
+; RV32I-NEXT: or a4, a4, a7
+; RV32I-NEXT: slli a7, a5, 1
; RV32I-NEXT: srl a3, a3, a2
-; RV32I-NEXT: sll a7, a7, a5
+; RV32I-NEXT: sll a7, a7, a6
; RV32I-NEXT: or a3, a3, a7
-; RV32I-NEXT: slli a7, a4, 1
+; RV32I-NEXT: slli a7, a1, 1
+; RV32I-NEXT: srl a5, a5, a2
+; RV32I-NEXT: sll a6, a7, a6
+; RV32I-NEXT: or a5, a5, a6
; RV32I-NEXT: srl a1, a1, a2
-; RV32I-NEXT: sll a7, a7, a5
-; RV32I-NEXT: or a1, a1, a7
-; RV32I-NEXT: slli a7, a6, 1
-; RV32I-NEXT: srl a4, a4, a2
-; RV32I-NEXT: sll a5, a7, a5
-; RV32I-NEXT: or a4, a4, a5
-; RV32I-NEXT: srl a2, a6, a2
-; RV32I-NEXT: sw a3, 0(a0)
-; RV32I-NEXT: sw a1, 4(a0)
-; RV32I-NEXT: sw a4, 8(a0)
-; RV32I-NEXT: sw a2, 12(a0)
+; RV32I-NEXT: sw a4, 0(a0)
+; RV32I-NEXT: sw a3, 4(a0)
+; RV32I-NEXT: sw a5, 8(a0)
+; RV32I-NEXT: sw a1, 12(a0)
; RV32I-NEXT: addi sp, sp, 32
; RV32I-NEXT: ret
;
@@ -235,7 +235,7 @@ define i128 @ashr128(i128 %a, i128 %b) nounwind {
; RV32I-NEXT: sw a1, 20(sp)
; RV32I-NEXT: sw a1, 24(sp)
; RV32I-NEXT: sw a1, 28(sp)
-; RV32I-NEXT: add a3, a4, a3
+; RV32I-NEXT: or a3, a4, a3
; RV32I-NEXT: lw a1, 4(a3)
; RV32I-NEXT: lw a4, 0(a3)
; RV32I-NEXT: lw a5, 8(a3)
@@ -588,30 +588,30 @@ define i128 @lshr128_shamt32(i128 %a, i32 signext %b) nounwind {
; RV32I-NEXT: sw a4, 4(sp)
; RV32I-NEXT: sw a5, 8(sp)
; RV32I-NEXT: sw a1, 12(sp)
-; RV32I-NEXT: add a6, a7, a6
-; RV32I-NEXT: lw a1, 4(a6)
-; RV32I-NEXT: lw a3, 0(a6)
-; RV32I-NEXT: lw a4, 8(a6)
-; RV32I-NEXT: andi a5, a2, 31
-; RV32I-NEXT: lw a6, 12(a6)
-; RV32I-NEXT: slli a7, a1, 1
-; RV32I-NEXT: xori a5, a5, 31
+; RV32I-NEXT: or a1, a7, a6
+; RV32I-NEXT: lw a3, 4(a1)
+; RV32I-NEXT: lw a4, 0(a1)
+; RV32I-NEXT: lw a5, 8(a1)
+; RV32I-NEXT: andi a6, a2, 31
+; RV32I-NEXT: lw a1, 12(a1)
+; RV32I-NEXT: slli a7, a3, 1
+; RV32I-NEXT: xori a6, a6, 31
+; RV32I-NEXT: srl a4, a4, a2
+; RV32I-NEXT: sll a7, a7, a6
+; RV32I-NEXT: or a4, a4, a7
+; RV32I-NEXT: slli a7, a5, 1
; RV32I-NEXT: srl a3, a3, a2
-; RV32I-NEXT: sll a7, a7, a5
+; RV32I-NEXT: sll a7, a7, a6
; RV32I-NEXT: or a3, a3, a7
-; RV32I-NEXT: slli a7, a4, 1
+; RV32I-NEXT: slli a7, a1, 1
+; RV32I-NEXT: srl a5, a5, a2
+; RV32I-NEXT: sll a6, a7, a6
+; RV32I-NEXT: or a5, a5, a6
; RV32I-NEXT: srl a1, a1, a2
-; RV32I-NEXT: sll a7, a7, a5
-; RV32I-NEXT: or a1, a1, a7
-; RV32I-NEXT: slli a7, a6, 1
-; RV32I-NEXT: srl a4, a4, a2
-; RV32I-NEXT: sll a5, a7, a5
-; RV32I-NEXT: or a4, a4, a5
-; RV32I-NEXT: srl a2, a6, a2
-; RV32I-NEXT: sw a3, 0(a0)
-; RV32I-NEXT: sw a1, 4(a0)
-; RV32I-NEXT: sw a4, 8(a0)
-; RV32I-NEXT: sw a2, 12(a0)
+; RV32I-NEXT: sw a4, 0(a0)
+; RV32I-NEXT: sw a3, 4(a0)
+; RV32I-NEXT: sw a5, 8(a0)
+; RV32I-NEXT: sw a1, 12(a0)
; RV32I-NEXT: addi sp, sp, 32
; RV32I-NEXT: ret
;
@@ -658,7 +658,7 @@ define i128 @ashr128_shamt32(i128 %a, i32 signext %b) nounwind {
; RV32I-NEXT: sw a1, 20(sp)
; RV32I-NEXT: sw a1, 24(sp)
; RV32I-NEXT: sw a1, 28(sp)
-; RV32I-NEXT: add a3, a4, a3
+; RV32I-NEXT: or a3, a4, a3
; RV32I-NEXT: lw a1, 4(a3)
; RV32I-NEXT: lw a4, 0(a3)
; RV32I-NEXT: lw a5, 8(a3)
diff --git a/llvm/test/CodeGen/RISCV/stack-store-check.ll b/llvm/test/CodeGen/RISCV/stack-store-check.ll
index 6bcee259896f6..42edef297cf22 100644
--- a/llvm/test/CodeGen/RISCV/stack-store-check.ll
+++ b/llvm/test/CodeGen/RISCV/stack-store-check.ll
@@ -33,256 +33,256 @@ define void @main() local_unnamed_addr nounwind {
; CHECK-NEXT: lw s10, %lo(U+4)(a0)
; CHECK-NEXT: lw s11, %lo(U+8)(a0)
; CHECK-NEXT: lw s5, %lo(U+12)(a0)
+; CHECK-NEXT: sw zero, 608(sp)
+; CHECK-NEXT: sw zero, 612(sp)
; CHECK-NEXT: sw zero, 616(sp)
; CHECK-NEXT: sw zero, 620(sp)
-; CHECK-NEXT: sw zero, 624(sp)
-; CHECK-NEXT: sw zero, 628(sp)
-; CHECK-NEXT: addi a0, sp, 632
-; CHECK-NEXT: addi a1, sp, 616
-; CHECK-NEXT: addi a2, sp, 600
-; CHECK-NEXT: sw s9, 600(sp)
-; CHECK-NEXT: sw s10, 604(sp)
-; CHECK-NEXT: sw s11, 608(sp)
-; CHECK-NEXT: sw s5, 612(sp)
+; CHECK-NEXT: addi a0, sp, 624
+; CHECK-NEXT: addi a1, sp, 608
+; CHECK-NEXT: addi a2, sp, 592
+; CHECK-NEXT: sw s9, 592(sp)
+; CHECK-NEXT: sw s10, 596(sp)
+; CHECK-NEXT: sw s11, 600(sp)
+; CHECK-NEXT: sw s5, 604(sp)
; CHECK-NEXT: call __subtf3
-; CHECK-NEXT: lw s1, 632(sp)
-; CHECK-NEXT: lw s2, 636(sp)
-; CHECK-NEXT: lw s3, 640(sp)
-; CHECK-NEXT: lw s4, 644(sp)
-; CHECK-NEXT: sw s9, 552(sp)
-; CHECK-NEXT: sw s10, 556(sp)
-; CHECK-NEXT: sw s11, 560(sp)
-; CHECK-NEXT: sw s5, 564(sp)
-; CHECK-NEXT: addi a0, sp, 584
-; CHECK-NEXT: addi a1, sp, 568
-; CHECK-NEXT: addi a2, sp, 552
-; CHECK-NEXT: sw s1, 568(sp)
-; CHECK-NEXT: sw s2, 572(sp)
-; CHECK-NEXT: sw s3, 576(sp)
-; CHECK-NEXT: sw s4, 580(sp)
+; CHECK-NEXT: lw s1, 624(sp)
+; CHECK-NEXT: lw s2, 628(sp)
+; CHECK-NEXT: lw s3, 632(sp)
+; CHECK-NEXT: lw s4, 636(sp)
+; CHECK-NEXT: sw s9, 544(sp)
+; CHECK-NEXT: sw s10, 548(sp)
+; CHECK-NEXT: sw s11, 552(sp)
+; CHECK-NEXT: sw s5, 556(sp)
+; CHECK-NEXT: addi a0, sp, 576
+; CHECK-NEXT: addi a1, sp, 560
+; CHECK-NEXT: addi a2, sp, 544
+; CHECK-NEXT: sw s1, 560(sp)
+; CHECK-NEXT: sw s2, 564(sp)
+; CHECK-NEXT: sw s3, 568(sp)
+; CHECK-NEXT: sw s4, 572(sp)
; CHECK-NEXT: call __subtf3
-; CHECK-NEXT: lw a0, 584(sp)
-; CHECK-NEXT: sw a0, 52(sp) # 4-byte Folded Spill
-; CHECK-NEXT: lw a0, 588(sp)
-; CHECK-NEXT: sw a0, 48(sp) # 4-byte Folded Spill
-; CHECK-NEXT: lw a0, 592(sp)
+; CHECK-NEXT: lw a0, 576(sp)
; CHECK-NEXT: sw a0, 44(sp) # 4-byte Folded Spill
-; CHECK-NEXT: lw a0, 596(sp)
+; CHECK-NEXT: lw a0, 580(sp)
; CHECK-NEXT: sw a0, 40(sp) # 4-byte Folded Spill
+; CHECK-NEXT: lw a0, 584(sp)
+; CHECK-NEXT: sw a0, 36(sp) # 4-byte Folded Spill
+; CHECK-NEXT: lw a0, 588(sp)
+; CHECK-NEXT: sw a0, 32(sp) # 4-byte Folded Spill
+; CHECK-NEXT: sw zero, 496(sp)
+; CHECK-NEXT: sw zero, 500(sp)
; CHECK-NEXT: sw zero, 504(sp)
; CHECK-NEXT: sw zero, 508(sp)
-; CHECK-NEXT: sw zero, 512(sp)
-; CHECK-NEXT: sw zero, 516(sp)
-; CHECK-NEXT: addi a0, sp, 536
-; CHECK-NEXT: addi a1, sp, 520
-; CHECK-NEXT: addi a2, sp, 504
-; CHECK-NEXT: sw s9, 520(sp)
-; CHECK-NEXT: sw s10, 524(sp)
-; CHECK-NEXT: sw s11, 528(sp)
-; CHECK-NEXT: sw s5, 532(sp)
+; CHECK-NEXT: addi a0, sp, 528
+; CHECK-NEXT: addi a1, sp, 512
+; CHECK-NEXT: addi a2, sp, 496
+; CHECK-NEXT: sw s9, 512(sp)
+; CHECK-NEXT: sw s10, 516(sp)
+; CHECK-NEXT: sw s11, 520(sp)
+; CHECK-NEXT: sw s5, 524(sp)
; CHECK-NEXT: call __addtf3
-; CHECK-NEXT: lw s0, 536(sp)
-; CHECK-NEXT: lw s6, 540(sp)
-; CHECK-NEXT: lw s7, 544(sp)
-; CHECK-NEXT: lw s8, 548(sp)
+; CHECK-NEXT: lw s0, 528(sp)
+; CHECK-NEXT: lw s6, 532(sp)
+; CHECK-NEXT: lw s7, 536(sp)
+; CHECK-NEXT: lw s8, 540(sp)
; CHECK-NEXT: lui a0, %hi(Y1)
; CHECK-NEXT: lw a1, %lo(Y1)(a0)
-; CHECK-NEXT: sw a1, 20(sp) # 4-byte Folded Spill
+; CHECK-NEXT: sw a1, 12(sp) # 4-byte Folded Spill
; CHECK-NEXT: lw a2, %lo(Y1+4)(a0)
-; CHECK-NEXT: sw a2, 16(sp) # 4-byte Folded Spill
+; CHECK-NEXT: sw a2, 8(sp) # 4-byte Folded Spill
; CHECK-NEXT: lw a3, %lo(Y1+8)(a0)
-; CHECK-NEXT: sw a3, 12(sp) # 4-byte Folded Spill
+; CHECK-NEXT: sw a3, 4(sp) # 4-byte Folded Spill
; CHECK-NEXT: lw a0, %lo(Y1+12)(a0)
-; CHECK-NEXT: sw a0, 8(sp) # 4-byte Folded Spill
-; CHECK-NEXT: sw a1, 312(sp)
-; CHECK-NEXT: sw a2, 316(sp)
-; CHECK-NEXT: sw a3, 320(sp)
-; CHECK-NEXT: sw a0, 324(sp)
-; CHECK-NEXT: addi a0, sp, 344
-; CHECK-NEXT: addi a1, sp, 328
-; CHECK-NEXT: addi a2, sp, 312
-; CHECK-NEXT: sw s1, 328(sp)
-; CHECK-NEXT: sw s2, 332(sp)
-; CHECK-NEXT: sw s3, 336(sp)
-; CHECK-NEXT: sw s4, 340(sp)
+; CHECK-NEXT: sw a0, 0(sp) # 4-byte Folded Spill
+; CHECK-NEXT: sw a1, 304(sp)
+; CHECK-NEXT: sw a2, 308(sp)
+; CHECK-NEXT: sw a3, 312(sp)
+; CHECK-NEXT: sw a0, 316(sp)
+; CHECK-NEXT: addi a0, sp, 336
+; CHECK-NEXT: addi a1, sp, 320
+; CHECK-NEXT: addi a2, sp, 304
+; CHECK-NEXT: sw s1, 320(sp)
+; CHECK-NEXT: sw s2, 324(sp)
+; CHECK-NEXT: sw s3, 328(sp)
+; CHECK-NEXT: sw s4, 332(sp)
; CHECK-NEXT: call __multf3
-; CHECK-NEXT: lw a0, 344(sp)
-; CHECK-NEXT: sw a0, 68(sp) # 4-byte Folded Spill
-; CHECK-NEXT: lw a0, 348(sp)
-; CHECK-NEXT: sw a0, 64(sp) # 4-byte Folded Spill
-; CHECK-NEXT: lw a0, 352(sp)
+; CHECK-NEXT: lw a0, 336(sp)
; CHECK-NEXT: sw a0, 60(sp) # 4-byte Folded Spill
-; CHECK-NEXT: lw a0, 356(sp)
+; CHECK-NEXT: lw a0, 340(sp)
; CHECK-NEXT: sw a0, 56(sp) # 4-byte Folded Spill
-; CHECK-NEXT: sw s9, 472(sp)
-; CHECK-NEXT: sw s10, 476(sp)
-; CHECK-NEXT: sw s11, 480(sp)
-; CHECK-NEXT: sw s5, 484(sp)
-; CHECK-NEXT: addi a0, sp, 488
-; CHECK-NEXT: addi a1, sp, 472
-; CHECK-NEXT: addi a2, sp, 456
-; CHECK-NEXT: sw s0, 456(sp)
-; CHECK-NEXT: sw s6, 460(sp)
-; CHECK-NEXT: sw s7, 464(sp)
-; CHECK-NEXT: sw s8, 468(sp)
+; CHECK-NEXT: lw a0, 344(sp)
+; CHECK-NEXT: sw a0, 52(sp) # 4-byte Folded Spill
+; CHECK-NEXT: lw a0, 348(sp)
+; CHECK-NEXT: sw a0, 48(sp) # 4-byte Folded Spill
+; CHECK-NEXT: sw s9, 464(sp)
+; CHECK-NEXT: sw s10, 468(sp)
+; CHECK-NEXT: sw s11, 472(sp)
+; CHECK-NEXT: sw s5, 476(sp)
+; CHECK-NEXT: addi a0, sp, 480
+; CHECK-NEXT: addi a1, sp, 464
+; CHECK-NEXT: addi a2, sp, 448
+; CHECK-NEXT: sw s0, 448(sp)
+; CHECK-NEXT: sw s6, 452(sp)
+; CHECK-NEXT: sw s7, 456(sp)
+; CHECK-NEXT: sw s8, 460(sp)
; CHECK-NEXT: call __addtf3
-; CHECK-NEXT: lw a3, 488(sp)
-; CHECK-NEXT: lw a4, 492(sp)
-; CHECK-NEXT: lw a5, 496(sp)
-; CHECK-NEXT: lw a6, 500(sp)
+; CHECK-NEXT: lw a3, 480(sp)
+; CHECK-NEXT: lw a4, 484(sp)
+; CHECK-NEXT: lw a5, 488(sp)
+; CHECK-NEXT: lw a6, 492(sp)
+; CHECK-NEXT: sw zero, 416(sp)
+; CHECK-NEXT: sw zero, 420(sp)
; CHECK-NEXT: sw zero, 424(sp)
; CHECK-NEXT: sw zero, 428(sp)
-; CHECK-NEXT: sw zero, 432(sp)
-; CHECK-NEXT: sw zero, 436(sp)
-; CHECK-NEXT: addi a0, sp, 440
-; CHECK-NEXT: addi a1, sp, 424
-; CHECK-NEXT: addi a2, sp, 408
-; CHECK-NEXT: sw a3, 408(sp)
-; CHECK-NEXT: sw a4, 412(sp)
-; CHECK-NEXT: sw a5, 416(sp)
-; CHECK-NEXT: sw a6, 420(sp)
+; CHECK-NEXT: addi a0, sp, 432
+; CHECK-NEXT: addi a1, sp, 416
+; CHECK-NEXT: addi a2, sp, 400
+; CHECK-NEXT: sw a3, 400(sp)
+; CHECK-NEXT: sw a4, 404(sp)
+; CHECK-NEXT: sw a5, 408(sp)
+; CHECK-NEXT: sw a6, 412(sp)
; CHECK-NEXT: call __subtf3
-; CHECK-NEXT: lw a0, 448(sp)
-; CHECK-NEXT: lw a1, 452(sp)
-; CHECK-NEXT: lw a2, 440(sp)
-; CHECK-NEXT: lw a3, 444(sp)
+; CHECK-NEXT: lw a0, 440(sp)
+; CHECK-NEXT: lw a1, 444(sp)
+; CHECK-NEXT: lw a2, 432(sp)
+; CHECK-NEXT: lw a3, 436(sp)
; CHECK-NEXT: lui a4, %hi(X)
; CHECK-NEXT: sw a1, %lo(X+12)(a4)
; CHECK-NEXT: sw a0, %lo(X+8)(a4)
; CHECK-NEXT: sw a3, %lo(X+4)(a4)
; CHECK-NEXT: sw a2, %lo(X)(a4)
-; CHECK-NEXT: lw s5, 20(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw s5, 216(sp)
-; CHECK-NEXT: lw s9, 16(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw s9, 220(sp)
-; CHECK-NEXT: lw s10, 12(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw s10, 224(sp)
-; CHECK-NEXT: lw s11, 8(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw s11, 228(sp)
-; CHECK-NEXT: addi a0, sp, 248
-; CHECK-NEXT: addi a1, sp, 232
-; CHECK-NEXT: addi a2, sp, 216
-; CHECK-NEXT: lw s1, 52(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw s1, 232(sp)
-; CHECK-NEXT: lw s2, 48(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw s2, 236(sp)
-; CHECK-NEXT: lw s3, 44(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw s3, 240(sp)
-; CHECK-NEXT: lw s4, 40(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw s4, 244(sp)
+; CHECK-NEXT: lw s5, 12(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw s5, 208(sp)
+; CHECK-NEXT: lw s9, 8(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw s9, 212(sp)
+; CHECK-NEXT: lw s10, 4(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw s10, 216(sp)
+; CHECK-NEXT: lw s11, 0(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw s11, 220(sp)
+; CHECK-NEXT: addi a0, sp, 240
+; CHECK-NEXT: addi a1, sp, 224
+; CHECK-NEXT: addi a2, sp, 208
+; CHECK-NEXT: lw s1, 44(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw s1, 224(sp)
+; CHECK-NEXT: lw s2, 40(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw s2, 228(sp)
+; CHECK-NEXT: lw s3, 36(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw s3, 232(sp)
+; CHECK-NEXT: lw s4, 32(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw s4, 236(sp)
; CHECK-NEXT: call __multf3
-; CHECK-NEXT: lw a0, 248(sp)
-; CHECK-NEXT: sw a0, 36(sp) # 4-byte Folded Spill
-; CHECK-NEXT: lw a0, 252(sp)
-; CHECK-NEXT: sw a0, 32(sp) # 4-byte Folded Spill
-; CHECK-NEXT: lw a0, 256(sp)
+; CHECK-NEXT: lw a0, 240(sp)
; CHECK-NEXT: sw a0, 28(sp) # 4-byte Folded Spill
-; CHECK-NEXT: lw a0, 260(sp)
+; CHECK-NEXT: lw a0, 244(sp)
; CHECK-NEXT: sw a0, 24(sp) # 4-byte Folded Spill
+; CHECK-NEXT: lw a0, 248(sp)
+; CHECK-NEXT: sw a0, 20(sp) # 4-byte Folded Spill
+; CHECK-NEXT: lw a0, 252(sp)
+; CHECK-NEXT: sw a0, 16(sp) # 4-byte Folded Spill
+; CHECK-NEXT: sw zero, 352(sp)
+; CHECK-NEXT: sw zero, 356(sp)
; CHECK-NEXT: sw zero, 360(sp)
; CHECK-NEXT: sw zero, 364(sp)
-; CHECK-NEXT: sw zero, 368(sp)
-; CHECK-NEXT: sw zero, 372(sp)
-; CHECK-NEXT: addi a0, sp, 392
-; CHECK-NEXT: addi a1, sp, 376
-; CHECK-NEXT: addi a2, sp, 360
-; CHECK-NEXT: sw s0, 376(sp)
-; CHECK-NEXT: sw s6, 380(sp)
-; CHECK-NEXT: sw s7, 384(sp)
-; CHECK-NEXT: sw s8, 388(sp)
+; CHECK-NEXT: addi a0, sp, 384
+; CHECK-NEXT: addi a1, sp, 368
+; CHECK-NEXT: addi a2, sp, 352
+; CHECK-NEXT: sw s0, 368(sp)
+; CHECK-NEXT: sw s6, 372(sp)
+; CHECK-NEXT: sw s7, 376(sp)
+; CHECK-NEXT: sw s8, 380(sp)
; CHECK-NEXT: call __multf3
-; CHECK-NEXT: lw a0, 400(sp)
-; CHECK-NEXT: lw a1, 404(sp)
-; CHECK-NEXT: lw a2, 392(sp)
-; CHECK-NEXT: lw a3, 396(sp)
+; CHECK-NEXT: lw a0, 392(sp)
+; CHECK-NEXT: lw a1, 396(sp)
+; CHECK-NEXT: lw a2, 384(sp)
+; CHECK-NEXT: lw a3, 388(sp)
; CHECK-NEXT: lui a4, %hi(S)
; CHECK-NEXT: sw a1, %lo(S+12)(a4)
; CHECK-NEXT: sw a0, %lo(S+8)(a4)
; CHECK-NEXT: sw a3, %lo(S+4)(a4)
; CHECK-NEXT: sw a2, %lo(S)(a4)
-; CHECK-NEXT: sw s1, 264(sp)
-; CHECK-NEXT: sw s2, 268(sp)
-; CHECK-NEXT: sw s3, 272(sp)
-; CHECK-NEXT: sw s4, 276(sp)
-; CHECK-NEXT: addi a0, sp, 296
-; CHECK-NEXT: addi a1, sp, 280
-; CHECK-NEXT: addi a2, sp, 264
-; CHECK-NEXT: lw a3, 68(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw a3, 280(sp)
-; CHECK-NEXT: lw a3, 64(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw a3, 284(sp)
+; CHECK-NEXT: sw s1, 256(sp)
+; CHECK-NEXT: sw s2, 260(sp)
+; CHECK-NEXT: sw s3, 264(sp)
+; CHECK-NEXT: sw s4, 268(sp)
+; CHECK-NEXT: addi a0, sp, 288
+; CHECK-NEXT: addi a1, sp, 272
+; CHECK-NEXT: addi a2, sp, 256
; CHECK-NEXT: lw a3, 60(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw a3, 288(sp)
+; CHECK-NEXT: sw a3, 272(sp)
; CHECK-NEXT: lw a3, 56(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw a3, 292(sp)
+; CHECK-NEXT: sw a3, 276(sp)
+; CHECK-NEXT: lw a3, 52(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw a3, 280(sp)
+; CHECK-NEXT: lw a3, 48(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw a3, 284(sp)
; CHECK-NEXT: call __subtf3
-; CHECK-NEXT: lw a0, 304(sp)
-; CHECK-NEXT: lw a1, 308(sp)
-; CHECK-NEXT: lw a2, 296(sp)
-; CHECK-NEXT: lw a3, 300(sp)
+; CHECK-NEXT: lw a0, 296(sp)
+; CHECK-NEXT: lw a1, 300(sp)
+; CHECK-NEXT: lw a2, 288(sp)
+; CHECK-NEXT: lw a3, 292(sp)
; CHECK-NEXT: lui a4, %hi(T)
; CHECK-NEXT: sw a1, %lo(T+12)(a4)
; CHECK-NEXT: sw a0, %lo(T+8)(a4)
; CHECK-NEXT: sw a3, %lo(T+4)(a4)
; CHECK-NEXT: sw a2, %lo(T)(a4)
+; CHECK-NEXT: sw zero, 160(sp)
+; CHECK-NEXT: sw zero, 164(sp)
; CHECK-NEXT: sw zero, 168(sp)
; CHECK-NEXT: sw zero, 172(sp)
-; CHECK-NEXT: sw zero, 176(sp)
-; CHECK-NEXT: sw zero, 180(sp)
-; CHECK-NEXT: addi a0, sp, 200
-; CHECK-NEXT: addi a1, sp, 184
-; CHECK-NEXT: addi a2, sp, 168
-; CHECK-NEXT: lw a3, 36(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw a3, 184(sp)
-; CHECK-NEXT: lw a3, 32(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw a3, 188(sp)
+; CHECK-NEXT: addi a0, sp, 192
+; CHECK-NEXT: addi a1, sp, 176
+; CHECK-NEXT: addi a2, sp, 160
; CHECK-NEXT: lw a3, 28(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw a3, 192(sp)
+; CHECK-NEXT: sw a3, 176(sp)
; CHECK-NEXT: lw a3, 24(sp) # 4-byte Folded Reload
-; CHECK-NEXT: sw a3, 196(sp)
+; CHECK-NEXT: sw a3, 180(sp)
+; CHECK-NEXT: lw a3, 20(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw a3, 184(sp)
+; CHECK-NEXT: lw a3, 16(sp) # 4-byte Folded Reload
+; CHECK-NEXT: sw a3, 188(sp)
; CHECK-NEXT: call __addtf3
-; CHECK-NEXT: lw a0, 208(sp)
-; CHECK-NEXT: lw a1, 212(sp)
-; CHECK-NEXT: lw a2, 200(sp)
-; CHECK-NEXT: lw a3, 204(sp)
+; CHECK-NEXT: lw a0, 200(sp)
+; CHECK-NEXT: lw a1, 204(sp)
+; CHECK-NEXT: lw a2, 192(sp)
+; CHECK-NEXT: lw a3, 196(sp)
; CHECK-NEXT: lui a4, %hi(Y)
; CHECK-NEXT: sw a1, %lo(Y+12)(a4)
; CHECK-NEXT: sw a0, %lo(Y+8)(a4)
; CHECK-NEXT: sw a3, %lo(Y+4)(a4)
; CHECK-NEXT: sw a2, %lo(Y)(a4)
+; CHECK-NEXT: sw zero, 112(sp)
+; CHECK-NEXT: sw zero, 116(sp)
; CHECK-NEXT: sw zero, 120(sp)
; CHECK-NEXT: sw zero, 124(sp)
-; CHECK-NEXT: sw zero, 128(sp)
-; CHECK-NEXT: sw zero, 132(sp)
-; CHECK-NEXT: addi a0, sp, 152
-; CHECK-NEXT: addi a1, sp, 136
-; CHECK-NEXT: addi a2, sp, 120
-; CHECK-NEXT: sw s5, 136(sp)
-; CHECK-NEXT: sw s9, 140(sp)
-; CHECK-NEXT: sw s10, 144(sp)
-; CHECK-NEXT: sw s11, 148(sp)
+; CHECK-NEXT: addi a0, sp, 144
+; CHECK-NEXT: addi a1, sp, 128
+; CHECK-NEXT: addi a2, sp, 112
+; CHECK-NEXT: sw s5, 128(sp)
+; CHECK-NEXT: sw s9, 132(sp)
+; CHECK-NEXT: sw s10, 136(sp)
+; CHECK-NEXT: sw s11, 140(sp)
; CHECK-NEXT: call __multf3
-; CHECK-NEXT: lw a3, 152(sp)
-; CHECK-NEXT: lw a4, 156(sp)
-; CHECK-NEXT: lw a5, 160(sp)
-; CHECK-NEXT: lw a6, 164(sp)
+; CHECK-NEXT: lw a3, 144(sp)
+; CHECK-NEXT: lw a4, 148(sp)
+; CHECK-NEXT: lw a5, 152(sp)
+; CHECK-NEXT: lw a6, 156(sp)
; CHECK-NEXT: lui a0, 786400
+; CHECK-NEXT: sw zero, 64(sp)
+; CHECK-NEXT: sw zero, 68(sp)
; CHECK-NEXT: sw zero, 72(sp)
-; CHECK-NEXT: sw zero, 76(sp)
-; CHECK-NEXT: sw zero, 80(sp)
-; CHECK-NEXT: sw a0, 84(sp)
-; CHECK-NEXT: addi a0, sp, 104
-; CHECK-NEXT: addi a1, sp, 88
-; CHECK-NEXT: addi a2, sp, 72
-; CHECK-NEXT: sw a3, 88(sp)
-; CHECK-NEXT: sw a4, 92(sp)
-; CHECK-NEXT: sw a5, 96(sp)
-; CHECK-NEXT: sw a6, 100(sp)
+; CHECK-NEXT: sw a0, 76(sp)
+; CHECK-NEXT: addi a0, sp, 96
+; CHECK-NEXT: addi a1, sp, 80
+; CHECK-NEXT: addi a2, sp, 64
+; CHECK-NEXT: sw a3, 80(sp)
+; CHECK-NEXT: sw a4, 84(sp)
+; CHECK-NEXT: sw a5, 88(sp)
+; CHECK-NEXT: sw a6, 92(sp)
; CHECK-NEXT: call __addtf3
-; CHECK-NEXT: lw a0, 112(sp)
-; CHECK-NEXT: lw a1, 116(sp)
-; CHECK-NEXT: lw a2, 108(sp)
-; CHECK-NEXT: lw a3, 104(sp)
+; CHECK-NEXT: lw a0, 104(sp)
+; CHECK-NEXT: lw a1, 108(sp)
+; CHECK-NEXT: lw a2, 100(sp)
+; CHECK-NEXT: lw a3, 96(sp)
; CHECK-NEXT: lui a4, %hi(Y1)
; CHECK-NEXT: sw a0, %lo(Y1+8)(a4)
; CHECK-NEXT: sw a1, %lo(Y1+12)(a4)
diff --git a/llvm/test/CodeGen/RISCV/wide-scalar-shift-by-byte-multiple-legalization.ll b/llvm/test/CodeGen/RISCV/wide-scalar-shift-by-byte-multiple-legalization.ll
index 7f8e17192ec01..e2155393a7f33 100644
--- a/llvm/test/CodeGen/RISCV/wide-scalar-shift-by-byte-multiple-legalization.ll
+++ b/llvm/test/CodeGen/RISCV/wide-scalar-shift-by-byte-multiple-legalization.ll
@@ -784,15 +784,15 @@ define void @lshr_16bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
; RV32I-NEXT: sw a5, 4(sp)
; RV32I-NEXT: sw a6, 8(sp)
; RV32I-NEXT: sw a0, 12(sp)
-; RV32I-NEXT: add a3, a7, a3
-; RV32I-NEXT: lw a0, 8(a3)
-; RV32I-NEXT: lw a4, 4(a3)
+; RV32I-NEXT: or a0, a7, a3
+; RV32I-NEXT: lw a3, 8(a0)
+; RV32I-NEXT: lw a4, 4(a0)
; RV32I-NEXT: slli a1, a1, 3
; RV32I-NEXT: andi a5, a1, 24
; RV32I-NEXT: xori a5, a5, 31
-; RV32I-NEXT: lw a6, 0(a3)
-; RV32I-NEXT: lw a3, 12(a3)
-; RV32I-NEXT: slli a7, a0, 1
+; RV32I-NEXT: lw a6, 0(a0)
+; RV32I-NEXT: lw a0, 12(a0)
+; RV32I-NEXT: slli a7, a3, 1
; RV32I-NEXT: srl t0, a4, a1
; RV32I-NEXT: sll a7, a7, a5
; RV32I-NEXT: or a7, t0, a7
@@ -800,25 +800,25 @@ define void @lshr_16bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
; RV32I-NEXT: srl a6, a6, a1
; RV32I-NEXT: sll a4, a4, a5
; RV32I-NEXT: or a4, a6, a4
-; RV32I-NEXT: slli t1, a3, 1
-; RV32I-NEXT: srl a0, a0, a1
+; RV32I-NEXT: slli t1, a0, 1
+; RV32I-NEXT: srl a3, a3, a1
; RV32I-NEXT: sll a5, t1, a5
-; RV32I-NEXT: or a5, a0, a5
-; RV32I-NEXT: srl a1, a3, a1
-; RV32I-NEXT: srli a3, a1, 16
-; RV32I-NEXT: srli t1, a1, 24
-; RV32I-NEXT: srli t2, a1, 8
-; RV32I-NEXT: sb a1, 12(a2)
+; RV32I-NEXT: or a5, a3, a5
+; RV32I-NEXT: srl a0, a0, a1
+; RV32I-NEXT: srli a1, a0, 16
+; RV32I-NEXT: srli t1, a0, 24
+; RV32I-NEXT: srli t2, a0, 8
+; RV32I-NEXT: sb a0, 12(a2)
; RV32I-NEXT: sb t2, 13(a2)
-; RV32I-NEXT: sb a3, 14(a2)
+; RV32I-NEXT: sb a1, 14(a2)
; RV32I-NEXT: sb t1, 15(a2)
-; RV32I-NEXT: srli a1, a5, 16
-; RV32I-NEXT: srli a3, a5, 24
+; RV32I-NEXT: srli a0, a5, 16
+; RV32I-NEXT: srli a1, a5, 24
; RV32I-NEXT: srli a5, a5, 8
-; RV32I-NEXT: sb a0, 8(a2)
+; RV32I-NEXT: sb a3, 8(a2)
; RV32I-NEXT: sb a5, 9(a2)
-; RV32I-NEXT: sb a1, 10(a2)
-; RV32I-NEXT: sb a3, 11(a2)
+; RV32I-NEXT: sb a0, 10(a2)
+; RV32I-NEXT: sb a1, 11(a2)
; RV32I-NEXT: srli a0, a4, 16
; RV32I-NEXT: srli a1, a4, 24
; RV32I-NEXT: srli a4, a4, 8
@@ -1015,24 +1015,24 @@ define void @lshr_16bytes_wordOff(ptr %src.ptr, ptr %wordOff.ptr, ptr %dst) noun
; RV32I-NEXT: sw a5, 4(sp)
; RV32I-NEXT: sw a4, 8(sp)
; RV32I-NEXT: sw a0, 12(sp)
-; RV32I-NEXT: add a1, a6, a1
-; RV32I-NEXT: lw a0, 8(a1)
-; RV32I-NEXT: lw a3, 4(a1)
-; RV32I-NEXT: lw a4, 0(a1)
-; RV32I-NEXT: lw a1, 12(a1)
-; RV32I-NEXT: srli a5, a0, 16
-; RV32I-NEXT: srli a6, a0, 24
-; RV32I-NEXT: srli a7, a0, 8
-; RV32I-NEXT: sb a0, 8(a2)
+; RV32I-NEXT: or a0, a6, a1
+; RV32I-NEXT: lw a1, 8(a0)
+; RV32I-NEXT: lw a3, 4(a0)
+; RV32I-NEXT: lw a4, 0(a0)
+; RV32I-NEXT: lw a0, 12(a0)
+; RV32I-NEXT: srli a5, a1, 16
+; RV32I-NEXT: srli a6, a1, 24
+; RV32I-NEXT: srli a7, a1, 8
+; RV32I-NEXT: sb a1, 8(a2)
; RV32I-NEXT: sb a7, 9(a2)
; RV32I-NEXT: sb a5, 10(a2)
; RV32I-NEXT: sb a6, 11(a2)
-; RV32I-NEXT: srli a0, a1, 16
-; RV32I-NEXT: srli a5, a1, 24
-; RV32I-NEXT: srli a6, a1, 8
-; RV32I-NEXT: sb a1, 12(a2)
+; RV32I-NEXT: srli a1, a0, 16
+; RV32I-NEXT: srli a5, a0, 24
+; RV32I-NEXT: srli a6, a0, 8
+; RV32I-NEXT: sb a0, 12(a2)
; RV32I-NEXT: sb a6, 13(a2)
-; RV32I-NEXT: sb a0, 14(a2)
+; RV32I-NEXT: sb a1, 14(a2)
; RV32I-NEXT: sb a5, 15(a2)
; RV32I-NEXT: srli a0, a4, 16
; RV32I-NEXT: srli a1, a4, 24
@@ -1694,7 +1694,7 @@ define void @ashr_16bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
; RV32I-NEXT: sw a5, 4(sp)
; RV32I-NEXT: sw a6, 8(sp)
; RV32I-NEXT: sw a7, 12(sp)
-; RV32I-NEXT: add a0, a3, a0
+; RV32I-NEXT: or a0, a3, a0
; RV32I-NEXT: lw a3, 8(a0)
; RV32I-NEXT: lw a4, 4(a0)
; RV32I-NEXT: slli a1, a1, 3
@@ -1926,7 +1926,7 @@ define void @ashr_16bytes_wordOff(ptr %src.ptr, ptr %wordOff.ptr, ptr %dst) noun
; RV32I-NEXT: sw a4, 4(sp)
; RV32I-NEXT: sw a5, 8(sp)
; RV32I-NEXT: sw a6, 12(sp)
-; RV32I-NEXT: add a0, a0, a1
+; RV32I-NEXT: or a0, a0, a1
; RV32I-NEXT: lw a1, 8(a0)
; RV32I-NEXT: lw a3, 4(a0)
; RV32I-NEXT: lw a4, 0(a0)
@@ -2700,25 +2700,25 @@ define void @lshr_32bytes_wordOff(ptr %src.ptr, ptr %wordOff.ptr, ptr %dst) noun
; RV32I-NEXT: or a0, a0, s1
; RV32I-NEXT: or t1, t2, t1
; RV32I-NEXT: or a0, a0, t3
+; RV32I-NEXT: sw zero, 48(sp)
+; RV32I-NEXT: sw zero, 52(sp)
; RV32I-NEXT: sw zero, 56(sp)
; RV32I-NEXT: sw zero, 60(sp)
-; RV32I-NEXT: sw zero, 64(sp)
-; RV32I-NEXT: sw zero, 68(sp)
+; RV32I-NEXT: sw zero, 32(sp)
+; RV32I-NEXT: sw zero, 36(sp)
; RV32I-NEXT: sw zero, 40(sp)
; RV32I-NEXT: sw zero, 44(sp)
-; RV32I-NEXT: sw zero, 48(sp)
-; RV32I-NEXT: sw zero, 52(sp)
; RV32I-NEXT: slli a1, a1, 2
-; RV32I-NEXT: sw a7, 24(sp)
-; RV32I-NEXT: sw t0, 28(sp)
-; RV32I-NEXT: sw t1, 32(sp)
-; RV32I-NEXT: sw a0, 36(sp)
+; RV32I-NEXT: sw a7, 16(sp)
+; RV32I-NEXT: sw t0, 20(sp)
+; RV32I-NEXT: sw t1, 24(sp)
+; RV32I-NEXT: sw a0, 28(sp)
; RV32I-NEXT: andi a1, a1, 28
-; RV32I-NEXT: addi a0, sp, 8
-; RV32I-NEXT: sw a3, 8(sp)
-; RV32I-NEXT: sw a4, 12(sp)
-; RV32I-NEXT: sw a5, 16(sp)
-; RV32I-NEXT: sw a6, 20(sp)
+; RV32I-NEXT: mv a0, sp
+; RV32I-NEXT: sw a3, 0(sp)
+; RV32I-NEXT: sw a4, 4(sp)
+; RV32I-NEXT: sw a5, 8(sp)
+; RV32I-NEXT: sw a6, 12(sp)
; RV32I-NEXT: add a4, a0, a1
; RV32I-NEXT: lw a5, 24(a4)
; RV32I-NEXT: lw a1, 0(a4)
@@ -3054,25 +3054,25 @@ define void @lshr_32bytes_dwordOff(ptr %src.ptr, ptr %dwordOff.ptr, ptr %dst) no
; RV32I-NEXT: or a0, a0, s1
; RV32I-NEXT: or t1, t2, t1
; RV32I-NEXT: or a0, a0, t3
+; RV32I-NEXT: sw zero, 48(sp)
+; RV32I-NEXT: sw zero, 52(sp)
; RV32I-NEXT: sw zero, 56(sp)
; RV32I-NEXT: sw zero, 60(sp)
-; RV32I-NEXT: sw zero, 64(sp)
-; RV32I-NEXT: sw zero, 68(sp)
+; RV32I-NEXT: sw zero, 32(sp)
+; RV32I-NEXT: sw zero, 36(sp)
; RV32I-NEXT: sw zero, 40(sp)
; RV32I-NEXT: sw zero, 44(sp)
-; RV32I-NEXT: sw zero, 48(sp)
-; RV32I-NEXT: sw zero, 52(sp)
; RV32I-NEXT: slli a1, a1, 3
-; RV32I-NEXT: sw a7, 24(sp)
-; RV32I-NEXT: sw t0, 28(sp)
-; RV32I-NEXT: sw t1, 32(sp)
-; RV32I-NEXT: sw a0, 36(sp)
+; RV32I-NEXT: sw a7, 16(sp)
+; RV32I-NEXT: sw t0, 20(sp)
+; RV32I-NEXT: sw t1, 24(sp)
+; RV32I-NEXT: sw a0, 28(sp)
; RV32I-NEXT: andi a1, a1, 24
-; RV32I-NEXT: addi a0, sp, 8
-; RV32I-NEXT: sw a3, 8(sp)
-; RV32I-NEXT: sw a4, 12(sp)
-; RV32I-NEXT: sw a5, 16(sp)
-; RV32I-NEXT: sw a6, 20(sp)
+; RV32I-NEXT: mv a0, sp
+; RV32I-NEXT: sw a3, 0(sp)
+; RV32I-NEXT: sw a4, 4(sp)
+; RV32I-NEXT: sw a5, 8(sp)
+; RV32I-NEXT: sw a6, 12(sp)
; RV32I-NEXT: add a4, a0, a1
; RV32I-NEXT: lw a5, 24(a4)
; RV32I-NEXT: lw a1, 0(a4)
@@ -3877,25 +3877,25 @@ define void @shl_32bytes_wordOff(ptr %src.ptr, ptr %wordOff.ptr, ptr %dst) nounw
; RV32I-NEXT: or a0, a0, s1
; RV32I-NEXT: or t1, t2, t1
; RV32I-NEXT: or a0, a0, t3
+; RV32I-NEXT: sw zero, 16(sp)
+; RV32I-NEXT: sw zero, 20(sp)
; RV32I-NEXT: sw zero, 24(sp)
; RV32I-NEXT: sw zero, 28(sp)
-; RV32I-NEXT: sw zero, 32(sp)
-; RV32I-NEXT: sw zero, 36(sp)
+; RV32I-NEXT: sw zero, 0(sp)
+; RV32I-NEXT: sw zero, 4(sp)
; RV32I-NEXT: sw zero, 8(sp)
; RV32I-NEXT: sw zero, 12(sp)
-; RV32I-NEXT: sw zero, 16(sp)
-; RV32I-NEXT: sw zero, 20(sp)
; RV32I-NEXT: slli a1, a1, 2
-; RV32I-NEXT: sw a7, 56(sp)
-; RV32I-NEXT: sw t0, 60(sp)
-; RV32I-NEXT: sw t1, 64(sp)
-; RV32I-NEXT: sw a0, 68(sp)
+; RV32I-NEXT: sw a7, 48(sp)
+; RV32I-NEXT: sw t0, 52(sp)
+; RV32I-NEXT: sw t1, 56(sp)
+; RV32I-NEXT: sw a0, 60(sp)
; RV32I-NEXT: andi a1, a1, 28
-; RV32I-NEXT: addi a0, sp, 40
-; RV32I-NEXT: sw a3, 40(sp)
-; RV32I-NEXT: sw a4, 44(sp)
-; RV32I-NEXT: sw a5, 48(sp)
-; RV32I-NEXT: sw a6, 52(sp)
+; RV32I-NEXT: addi a0, sp, 32
+; RV32I-NEXT: sw a3, 32(sp)
+; RV32I-NEXT: sw a4, 36(sp)
+; RV32I-NEXT: sw a5, 40(sp)
+; RV32I-NEXT: sw a6, 44(sp)
; RV32I-NEXT: sub a4, a0, a1
; RV32I-NEXT: lw a5, 24(a4)
; RV32I-NEXT: lw a1, 0(a4)
@@ -4231,25 +4231,25 @@ define void @shl_32bytes_dwordOff(ptr %src.ptr, ptr %dwordOff.ptr, ptr %dst) nou
; RV32I-NEXT: or a0, a0, s1
; RV32I-NEXT: or t1, t2, t1
; RV32I-NEXT: or a0, a0, t3
+; RV32I-NEXT: sw zero, 16(sp)
+; RV32I-NEXT: sw zero, 20(sp)
; RV32I-NEXT: sw zero, 24(sp)
; RV32I-NEXT: sw zero, 28(sp)
-; RV32I-NEXT: sw zero, 32(sp)
-; RV32I-NEXT: sw zero, 36(sp)
+; RV32I-NEXT: sw zero, 0(sp)
+; RV32I-NEXT: sw zero, 4(sp)
; RV32I-NEXT: sw zero, 8(sp)
; RV32I-NEXT: sw zero, 12(sp)
-; RV32I-NEXT: sw zero, 16(sp)
-; RV32I-NEXT: sw zero, 20(sp)
; RV32I-NEXT: slli a1, a1, 3
-; RV32I-NEXT: sw a7, 56(sp)
-; RV32I-NEXT: sw t0, 60(sp)
-; RV32I-NEXT: sw t1, 64(sp)
-; RV32I-NEXT: sw a0, 68(sp)
+; RV32I-NEXT: sw a7, 48(sp)
+; RV32I-NEXT: sw t0, 52(sp)
+; RV32I-NEXT: sw t1, 56(sp)
+; RV32I-NEXT: sw a0, 60(sp)
; RV32I-NEXT: andi a1, a1, 24
-; RV32I-NEXT: addi a0, sp, 40
-; RV32I-NEXT: sw a3, 40(sp)
-; RV32I-NEXT: sw a4, 44(sp)
-; RV32I-NEXT: sw a5, 48(sp)
-; RV32I-NEXT: sw a6, 52(sp)
+; RV32I-NEXT: addi a0, sp, 32
+; RV32I-NEXT: sw a3, 32(sp)
+; RV32I-NEXT: sw a4, 36(sp)
+; RV32I-NEXT: sw a5, 40(sp)
+; RV32I-NEXT: sw a6, 44(sp)
; RV32I-NEXT: sub a4, a0, a1
; RV32I-NEXT: lw a5, 24(a4)
; RV32I-NEXT: lw a1, 0(a4)
@@ -5063,25 +5063,25 @@ define void @ashr_32bytes_wordOff(ptr %src.ptr, ptr %wordOff.ptr, ptr %dst) noun
; RV32I-NEXT: or s0, a0, s0
; RV32I-NEXT: or t2, s0, t2
; RV32I-NEXT: srai a0, a0, 31
+; RV32I-NEXT: sw a0, 48(sp)
+; RV32I-NEXT: sw a0, 52(sp)
; RV32I-NEXT: sw a0, 56(sp)
; RV32I-NEXT: sw a0, 60(sp)
-; RV32I-NEXT: sw a0, 64(sp)
-; RV32I-NEXT: sw a0, 68(sp)
+; RV32I-NEXT: sw a0, 32(sp)
+; RV32I-NEXT: sw a0, 36(sp)
; RV32I-NEXT: sw a0, 40(sp)
; RV32I-NEXT: sw a0, 44(sp)
-; RV32I-NEXT: sw a0, 48(sp)
-; RV32I-NEXT: sw a0, 52(sp)
; RV32I-NEXT: slli a1, a1, 2
-; RV32I-NEXT: sw a7, 24(sp)
-; RV32I-NEXT: sw t0, 28(sp)
-; RV32I-NEXT: sw t1, 32(sp)
-; RV32I-NEXT: sw t2, 36(sp)
+; RV32I-NEXT: sw a7, 16(sp)
+; RV32I-NEXT: sw t0, 20(sp)
+; RV32I-NEXT: sw t1, 24(sp)
+; RV32I-NEXT: sw t2, 28(sp)
; RV32I-NEXT: andi a1, a1, 28
-; RV32I-NEXT: addi a0, sp, 8
-; RV32I-NEXT: sw a3, 8(sp)
-; RV32I-NEXT: sw a4, 12(sp)
-; RV32I-NEXT: sw a5, 16(sp)
-; RV32I-NEXT: sw a6, 20(sp)
+; RV32I-NEXT: mv a0, sp
+; RV32I-NEXT: sw a3, 0(sp)
+; RV32I-NEXT: sw a4, 4(sp)
+; RV32I-NEXT: sw a5, 8(sp)
+; RV32I-NEXT: sw a6, 12(sp)
; RV32I-NEXT: add a4, a0, a1
; RV32I-NEXT: lw a5, 24(a4)
; RV32I-NEXT: lw a1, 0(a4)
@@ -5417,25 +5417,25 @@ define void @ashr_32bytes_dwordOff(ptr %src.ptr, ptr %dwordOff.ptr, ptr %dst) no
; RV32I-NEXT: or s0, a0, s0
; RV32I-NEXT: or t2, s0, t2
; RV32I-NEXT: srai a0, a0, 31
+; RV32I-NEXT: sw a0, 48(sp)
+; RV32I-NEXT: sw a0, 52(sp)
; RV32I-NEXT: sw a0, 56(sp)
; RV32I-NEXT: sw a0, 60(sp)
-; RV32I-NEXT: sw a0, 64(sp)
-; RV32I-NEXT: sw a0, 68(sp)
+; RV32I-NEXT: sw a0, 32(sp)
+; RV32I-NEXT: sw a0, 36(sp)
; RV32I-NEXT: sw a0, 40(sp)
; RV32I-NEXT: sw a0, 44(sp)
-; RV32I-NEXT: sw a0, 48(sp)
-; RV32I-NEXT: sw a0, 52(sp)
; RV32I-NEXT: slli a1, a1, 3
-; RV32I-NEXT: sw a7, 24(sp)
-; RV32I-NEXT: sw t0, 28(sp)
-; RV32I-NEXT: sw t1, 32(sp)
-; RV32I-NEXT: sw t2, 36(sp)
+; RV32I-NEXT: sw a7, 16(sp)
+; RV32I-NEXT: sw t0, 20(sp)
+; RV32I-NEXT: sw t1, 24(sp)
+; RV32I-NEXT: sw t2, 28(sp)
; RV32I-NEXT: andi a1, a1, 24
-; RV32I-NEXT: addi a0, sp, 8
-; RV32I-NEXT: sw a3, 8(sp)
-; RV32I-NEXT: sw a4, 12(sp)
-; RV32I-NEXT: sw a5, 16(sp)
-; RV32I-NEXT: sw a6, 20(sp)
+; RV32I-NEXT: mv a0, sp
+; RV32I-NEXT: sw a3, 0(sp)
+; RV32I-NEXT: sw a4, 4(sp)
+; RV32I-NEXT: sw a5, 8(sp)
+; RV32I-NEXT: sw a6, 12(sp)
; RV32I-NEXT: add a4, a0, a1
; RV32I-NEXT: lw a5, 24(a4)
; RV32I-NEXT: lw a1, 0(a4)
diff --git a/llvm/test/CodeGen/RISCV/wide-scalar-shift-legalization.ll b/llvm/test/CodeGen/RISCV/wide-scalar-shift-legalization.ll
index 96d0978223e5b..8b901310d9e54 100644
--- a/llvm/test/CodeGen/RISCV/wide-scalar-shift-legalization.ll
+++ b/llvm/test/CodeGen/RISCV/wide-scalar-shift-legalization.ll
@@ -766,14 +766,14 @@ define void @lshr_16bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
; RV32I-NEXT: sw a5, 4(sp)
; RV32I-NEXT: sw a6, 8(sp)
; RV32I-NEXT: sw a0, 12(sp)
-; RV32I-NEXT: add a3, a7, a3
-; RV32I-NEXT: lw a0, 8(a3)
-; RV32I-NEXT: lw a4, 4(a3)
+; RV32I-NEXT: or a0, a7, a3
+; RV32I-NEXT: lw a3, 8(a0)
+; RV32I-NEXT: lw a4, 4(a0)
; RV32I-NEXT: andi a5, a1, 31
; RV32I-NEXT: xori a5, a5, 31
-; RV32I-NEXT: lw a6, 0(a3)
-; RV32I-NEXT: lw a3, 12(a3)
-; RV32I-NEXT: slli a7, a0, 1
+; RV32I-NEXT: lw a6, 0(a0)
+; RV32I-NEXT: lw a0, 12(a0)
+; RV32I-NEXT: slli a7, a3, 1
; RV32I-NEXT: srl t0, a4, a1
; RV32I-NEXT: sll a7, a7, a5
; RV32I-NEXT: or a7, t0, a7
@@ -781,25 +781,25 @@ define void @lshr_16bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
; RV32I-NEXT: srl a6, a6, a1
; RV32I-NEXT: sll a4, a4, a5
; RV32I-NEXT: or a4, a6, a4
-; RV32I-NEXT: slli a6, a3, 1
-; RV32I-NEXT: srl a0, a0, a1
+; RV32I-NEXT: slli a6, a0, 1
+; RV32I-NEXT: srl a3, a3, a1
; RV32I-NEXT: sll a5, a6, a5
-; RV32I-NEXT: or a0, a0, a5
-; RV32I-NEXT: srl a1, a3, a1
-; RV32I-NEXT: srli a3, a1, 16
-; RV32I-NEXT: srli a5, a1, 24
-; RV32I-NEXT: srli a6, a1, 8
-; RV32I-NEXT: sb a1, 12(a2)
+; RV32I-NEXT: or a3, a3, a5
+; RV32I-NEXT: srl a0, a0, a1
+; RV32I-NEXT: srli a1, a0, 16
+; RV32I-NEXT: srli a5, a0, 24
+; RV32I-NEXT: srli a6, a0, 8
+; RV32I-NEXT: sb a0, 12(a2)
; RV32I-NEXT: sb a6, 13(a2)
-; RV32I-NEXT: sb a3, 14(a2)
+; RV32I-NEXT: sb a1, 14(a2)
; RV32I-NEXT: sb a5, 15(a2)
-; RV32I-NEXT: srli a1, a0, 16
-; RV32I-NEXT: srli a3, a0, 24
-; RV32I-NEXT: srli a5, a0, 8
-; RV32I-NEXT: sb a0, 8(a2)
+; RV32I-NEXT: srli a0, a3, 16
+; RV32I-NEXT: srli a1, a3, 24
+; RV32I-NEXT: srli a5, a3, 8
+; RV32I-NEXT: sb a3, 8(a2)
; RV32I-NEXT: sb a5, 9(a2)
-; RV32I-NEXT: sb a1, 10(a2)
-; RV32I-NEXT: sb a3, 11(a2)
+; RV32I-NEXT: sb a0, 10(a2)
+; RV32I-NEXT: sb a1, 11(a2)
; RV32I-NEXT: srli a0, a4, 16
; RV32I-NEXT: srli a1, a4, 24
; RV32I-NEXT: srli a3, a4, 8
@@ -1239,7 +1239,7 @@ define void @ashr_16bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
; RV32I-NEXT: sw a5, 4(sp)
; RV32I-NEXT: sw a6, 8(sp)
; RV32I-NEXT: sw t1, 12(sp)
-; RV32I-NEXT: add a0, a0, a4
+; RV32I-NEXT: or a0, a0, a4
; RV32I-NEXT: lw a3, 8(a0)
; RV32I-NEXT: lw a4, 4(a0)
; RV32I-NEXT: andi a5, a1, 31
@@ -1599,25 +1599,25 @@ define void @lshr_32bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
; RV32I-NEXT: or a1, a1, s1
; RV32I-NEXT: or a0, a0, t3
; RV32I-NEXT: or a4, a1, a4
+; RV32I-NEXT: sw zero, 48(sp)
+; RV32I-NEXT: sw zero, 52(sp)
; RV32I-NEXT: sw zero, 56(sp)
; RV32I-NEXT: sw zero, 60(sp)
-; RV32I-NEXT: sw zero, 64(sp)
-; RV32I-NEXT: sw zero, 68(sp)
+; RV32I-NEXT: sw zero, 32(sp)
+; RV32I-NEXT: sw zero, 36(sp)
; RV32I-NEXT: sw zero, 40(sp)
; RV32I-NEXT: sw zero, 44(sp)
-; RV32I-NEXT: sw zero, 48(sp)
-; RV32I-NEXT: sw zero, 52(sp)
; RV32I-NEXT: srli a1, a4, 3
-; RV32I-NEXT: sw t0, 24(sp)
-; RV32I-NEXT: sw t1, 28(sp)
-; RV32I-NEXT: sw t2, 32(sp)
-; RV32I-NEXT: sw a0, 36(sp)
+; RV32I-NEXT: sw t0, 16(sp)
+; RV32I-NEXT: sw t1, 20(sp)
+; RV32I-NEXT: sw t2, 24(sp)
+; RV32I-NEXT: sw a0, 28(sp)
; RV32I-NEXT: andi a1, a1, 28
-; RV32I-NEXT: addi a0, sp, 8
-; RV32I-NEXT: sw a3, 8(sp)
-; RV32I-NEXT: sw a5, 12(sp)
-; RV32I-NEXT: sw a6, 16(sp)
-; RV32I-NEXT: sw a7, 20(sp)
+; RV32I-NEXT: mv a0, sp
+; RV32I-NEXT: sw a3, 0(sp)
+; RV32I-NEXT: sw a5, 4(sp)
+; RV32I-NEXT: sw a6, 8(sp)
+; RV32I-NEXT: sw a7, 12(sp)
; RV32I-NEXT: add a3, a0, a1
; RV32I-NEXT: lw a5, 8(a3)
; RV32I-NEXT: lw a0, 4(a3)
@@ -2027,25 +2027,25 @@ define void @shl_32bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
; RV32I-NEXT: or a1, a1, s1
; RV32I-NEXT: or a0, a0, t3
; RV32I-NEXT: or a4, a1, a4
+; RV32I-NEXT: sw zero, 16(sp)
+; RV32I-NEXT: sw zero, 20(sp)
; RV32I-NEXT: sw zero, 24(sp)
; RV32I-NEXT: sw zero, 28(sp)
-; RV32I-NEXT: sw zero, 32(sp)
-; RV32I-NEXT: sw zero, 36(sp)
+; RV32I-NEXT: sw zero, 0(sp)
+; RV32I-NEXT: sw zero, 4(sp)
; RV32I-NEXT: sw zero, 8(sp)
; RV32I-NEXT: sw zero, 12(sp)
-; RV32I-NEXT: sw zero, 16(sp)
-; RV32I-NEXT: sw zero, 20(sp)
; RV32I-NEXT: srli a1, a4, 3
-; RV32I-NEXT: sw t0, 56(sp)
-; RV32I-NEXT: sw t1, 60(sp)
-; RV32I-NEXT: sw t2, 64(sp)
-; RV32I-NEXT: sw a0, 68(sp)
+; RV32I-NEXT: sw t0, 48(sp)
+; RV32I-NEXT: sw t1, 52(sp)
+; RV32I-NEXT: sw t2, 56(sp)
+; RV32I-NEXT: sw a0, 60(sp)
; RV32I-NEXT: andi a1, a1, 28
-; RV32I-NEXT: addi a0, sp, 40
-; RV32I-NEXT: sw a3, 40(sp)
-; RV32I-NEXT: sw a5, 44(sp)
-; RV32I-NEXT: sw a6, 48(sp)
-; RV32I-NEXT: sw a7, 52(sp)
+; RV32I-NEXT: addi a0, sp, 32
+; RV32I-NEXT: sw a3, 32(sp)
+; RV32I-NEXT: sw a5, 36(sp)
+; RV32I-NEXT: sw a6, 40(sp)
+; RV32I-NEXT: sw a7, 44(sp)
; RV32I-NEXT: sub a5, a0, a1
; RV32I-NEXT: lw a6, 0(a5)
; RV32I-NEXT: lw a3, 4(a5)
@@ -2458,25 +2458,25 @@ define void @ashr_32bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
; RV32I-NEXT: or a1, a1, t4
; RV32I-NEXT: or a4, a1, a4
; RV32I-NEXT: srai a0, a0, 31
+; RV32I-NEXT: sw a0, 48(sp)
+; RV32I-NEXT: sw a0, 52(sp)
; RV32I-NEXT: sw a0, 56(sp)
; RV32I-NEXT: sw a0, 60(sp)
-; RV32I-NEXT: sw a0, 64(sp)
-; RV32I-NEXT: sw a0, 68(sp)
+; RV32I-NEXT: sw a0, 32(sp)
+; RV32I-NEXT: sw a0, 36(sp)
; RV32I-NEXT: sw a0, 40(sp)
; RV32I-NEXT: sw a0, 44(sp)
-; RV32I-NEXT: sw a0, 48(sp)
-; RV32I-NEXT: sw a0, 52(sp)
; RV32I-NEXT: srli a0, a4, 3
-; RV32I-NEXT: sw t0, 24(sp)
-; RV32I-NEXT: sw t1, 28(sp)
-; RV32I-NEXT: sw t2, 32(sp)
-; RV32I-NEXT: sw t5, 36(sp)
+; RV32I-NEXT: sw t0, 16(sp)
+; RV32I-NEXT: sw t1, 20(sp)
+; RV32I-NEXT: sw t2, 24(sp)
+; RV32I-NEXT: sw t5, 28(sp)
; RV32I-NEXT: andi a0, a0, 28
-; RV32I-NEXT: addi a1, sp, 8
-; RV32I-NEXT: sw a3, 8(sp)
-; RV32I-NEXT: sw a5, 12(sp)
-; RV32I-NEXT: sw a6, 16(sp)
-; RV32I-NEXT: sw a7, 20(sp)
+; RV32I-NEXT: mv a1, sp
+; RV32I-NEXT: sw a3, 0(sp)
+; RV32I-NEXT: sw a5, 4(sp)
+; RV32I-NEXT: sw a6, 8(sp)
+; RV32I-NEXT: sw a7, 12(sp)
; RV32I-NEXT: add a3, a1, a0
; RV32I-NEXT: lw a5, 8(a3)
; RV32I-NEXT: lw a0, 4(a3)
diff --git a/llvm/test/CodeGen/Thumb/umulo-128-legalisation-lowering.ll b/llvm/test/CodeGen/Thumb/umulo-128-legalisation-lowering.ll
index 8550f39e38fa5..cf7f3111a5fc0 100644
--- a/llvm/test/CodeGen/Thumb/umulo-128-legalisation-lowering.ll
+++ b/llvm/test/CodeGen/Thumb/umulo-128-legalisation-lowering.ll
@@ -8,38 +8,38 @@ define { i128, i8 } @muloti_test(i128 %l, i128 %r) unnamed_addr #0 {
; THUMBV6-NEXT: push {r4, r5, r6, r7, lr}
; THUMBV6-NEXT: .pad #60
; THUMBV6-NEXT: sub sp, #60
-; THUMBV6-NEXT: mov r6, r3
-; THUMBV6-NEXT: mov r1, r2
-; THUMBV6-NEXT: str r2, [sp, #36] @ 4-byte Spill
+; THUMBV6-NEXT: str r3, [sp, #56] @ 4-byte Spill
+; THUMBV6-NEXT: mov r5, r2
+; THUMBV6-NEXT: str r1, [sp, #36] @ 4-byte Spill
; THUMBV6-NEXT: mov r4, r0
; THUMBV6-NEXT: str r0, [sp, #48] @ 4-byte Spill
-; THUMBV6-NEXT: ldr r2, [sp, #88]
-; THUMBV6-NEXT: str r2, [sp, #56] @ 4-byte Spill
-; THUMBV6-NEXT: movs r5, #0
+; THUMBV6-NEXT: ldr r2, [sp, #84]
+; THUMBV6-NEXT: str r2, [sp, #32] @ 4-byte Spill
+; THUMBV6-NEXT: movs r6, #0
; THUMBV6-NEXT: mov r0, r1
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
-; THUMBV6-NEXT: str r1, [sp, #28] @ 4-byte Spill
+; THUMBV6-NEXT: str r1, [sp, #20] @ 4-byte Spill
; THUMBV6-NEXT: str r0, [r4]
-; THUMBV6-NEXT: ldr r2, [sp, #96]
+; THUMBV6-NEXT: ldr r2, [sp, #92]
; THUMBV6-NEXT: str r2, [sp, #40] @ 4-byte Spill
-; THUMBV6-NEXT: mov r4, r6
-; THUMBV6-NEXT: str r6, [sp, #44] @ 4-byte Spill
-; THUMBV6-NEXT: mov r0, r6
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: mov r4, r5
+; THUMBV6-NEXT: str r5, [sp, #44] @ 4-byte Spill
+; THUMBV6-NEXT: mov r0, r5
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
; THUMBV6-NEXT: str r0, [sp, #52] @ 4-byte Spill
; THUMBV6-NEXT: mov r7, r1
; THUMBV6-NEXT: subs r0, r1, #1
; THUMBV6-NEXT: sbcs r7, r0
-; THUMBV6-NEXT: ldr r0, [sp, #100]
-; THUMBV6-NEXT: str r0, [sp, #32] @ 4-byte Spill
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: ldr r6, [sp, #36] @ 4-byte Reload
-; THUMBV6-NEXT: mov r2, r6
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: ldr r0, [sp, #96]
+; THUMBV6-NEXT: str r0, [sp, #28] @ 4-byte Spill
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: ldr r5, [sp, #36] @ 4-byte Reload
+; THUMBV6-NEXT: mov r2, r5
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
; THUMBV6-NEXT: str r0, [sp, #24] @ 4-byte Spill
; THUMBV6-NEXT: subs r2, r1, #1
@@ -47,7 +47,7 @@ define { i128, i8 } @muloti_test(i128 %l, i128 %r) unnamed_addr #0 {
; THUMBV6-NEXT: subs r2, r4, #1
; THUMBV6-NEXT: mov r3, r4
; THUMBV6-NEXT: sbcs r3, r2
-; THUMBV6-NEXT: ldr r4, [sp, #32] @ 4-byte Reload
+; THUMBV6-NEXT: ldr r4, [sp, #28] @ 4-byte Reload
; THUMBV6-NEXT: subs r2, r4, #1
; THUMBV6-NEXT: sbcs r4, r2
; THUMBV6-NEXT: ands r4, r3
@@ -57,116 +57,124 @@ define { i128, i8 } @muloti_test(i128 %l, i128 %r) unnamed_addr #0 {
; THUMBV6-NEXT: ldr r1, [sp, #24] @ 4-byte Reload
; THUMBV6-NEXT: adds r7, r1, r0
; THUMBV6-NEXT: ldr r0, [sp, #40] @ 4-byte Reload
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: mov r2, r6
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: mov r2, r5
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
-; THUMBV6-NEXT: str r0, [sp, #24] @ 4-byte Spill
+; THUMBV6-NEXT: str r0, [sp, #16] @ 4-byte Spill
; THUMBV6-NEXT: adds r0, r1, r7
-; THUMBV6-NEXT: str r0, [sp, #20] @ 4-byte Spill
-; THUMBV6-NEXT: mov r0, r5
-; THUMBV6-NEXT: adcs r0, r5
+; THUMBV6-NEXT: str r0, [sp, #12] @ 4-byte Spill
+; THUMBV6-NEXT: mov r0, r6
+; THUMBV6-NEXT: adcs r0, r6
; THUMBV6-NEXT: orrs r0, r4
-; THUMBV6-NEXT: str r0, [sp, #16] @ 4-byte Spill
-; THUMBV6-NEXT: ldr r0, [sp, #92]
-; THUMBV6-NEXT: str r0, [sp, #52] @ 4-byte Spill
-; THUMBV6-NEXT: ldr r7, [sp, #80]
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: mov r2, r7
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: str r0, [sp, #8] @ 4-byte Spill
+; THUMBV6-NEXT: ldr r0, [sp, #88]
+; THUMBV6-NEXT: str r0, [sp, #24] @ 4-byte Spill
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: ldr r2, [sp, #56] @ 4-byte Reload
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
-; THUMBV6-NEXT: str r0, [sp, #12] @ 4-byte Spill
+; THUMBV6-NEXT: str r0, [sp, #52] @ 4-byte Spill
; THUMBV6-NEXT: mov r4, r1
; THUMBV6-NEXT: subs r0, r1, #1
; THUMBV6-NEXT: sbcs r4, r0
-; THUMBV6-NEXT: ldr r6, [sp, #84]
-; THUMBV6-NEXT: mov r0, r6
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: ldr r2, [sp, #56] @ 4-byte Reload
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: ldr r7, [sp, #80]
+; THUMBV6-NEXT: mov r0, r7
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: ldr r5, [sp, #32] @ 4-byte Reload
+; THUMBV6-NEXT: mov r2, r5
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
-; THUMBV6-NEXT: str r0, [sp, #4] @ 4-byte Spill
+; THUMBV6-NEXT: str r0, [sp] @ 4-byte Spill
; THUMBV6-NEXT: subs r2, r1, #1
; THUMBV6-NEXT: sbcs r1, r2
-; THUMBV6-NEXT: ldr r3, [sp, #52] @ 4-byte Reload
+; THUMBV6-NEXT: ldr r3, [sp, #24] @ 4-byte Reload
; THUMBV6-NEXT: subs r2, r3, #1
; THUMBV6-NEXT: sbcs r3, r2
-; THUMBV6-NEXT: str r6, [sp, #8] @ 4-byte Spill
-; THUMBV6-NEXT: subs r2, r6, #1
-; THUMBV6-NEXT: sbcs r6, r2
-; THUMBV6-NEXT: ands r6, r3
-; THUMBV6-NEXT: orrs r6, r1
-; THUMBV6-NEXT: orrs r6, r4
-; THUMBV6-NEXT: ldr r0, [sp, #12] @ 4-byte Reload
-; THUMBV6-NEXT: ldr r1, [sp, #4] @ 4-byte Reload
-; THUMBV6-NEXT: adds r4, r1, r0
-; THUMBV6-NEXT: mov r0, r7
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: ldr r2, [sp, #56] @ 4-byte Reload
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: str r7, [sp, #4] @ 4-byte Spill
+; THUMBV6-NEXT: subs r2, r7, #1
+; THUMBV6-NEXT: sbcs r7, r2
+; THUMBV6-NEXT: ands r7, r3
+; THUMBV6-NEXT: orrs r7, r1
+; THUMBV6-NEXT: orrs r7, r4
+; THUMBV6-NEXT: ldr r0, [sp, #52] @ 4-byte Reload
+; THUMBV6-NEXT: ldr r1, [sp] @ 4-byte Reload
+; THUMBV6-NEXT: adds r0, r1, r0
+; THUMBV6-NEXT: str r0, [sp, #52] @ 4-byte Spill
+; THUMBV6-NEXT: ldr r4, [sp, #56] @ 4-byte Reload
+; THUMBV6-NEXT: mov r0, r4
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: mov r2, r5
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
-; THUMBV6-NEXT: adds r4, r1, r4
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: adcs r1, r5
-; THUMBV6-NEXT: orrs r1, r6
+; THUMBV6-NEXT: ldr r2, [sp, #52] @ 4-byte Reload
+; THUMBV6-NEXT: adds r1, r1, r2
+; THUMBV6-NEXT: str r1, [sp, #52] @ 4-byte Spill
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: adcs r1, r6
+; THUMBV6-NEXT: orrs r1, r7
; THUMBV6-NEXT: ldr r3, [sp, #40] @ 4-byte Reload
-; THUMBV6-NEXT: ldr r2, [sp, #32] @ 4-byte Reload
+; THUMBV6-NEXT: ldr r2, [sp, #28] @ 4-byte Reload
; THUMBV6-NEXT: orrs r3, r2
; THUMBV6-NEXT: subs r2, r3, #1
; THUMBV6-NEXT: sbcs r3, r2
-; THUMBV6-NEXT: ldr r2, [sp, #8] @ 4-byte Reload
-; THUMBV6-NEXT: orrs r7, r2
-; THUMBV6-NEXT: subs r2, r7, #1
-; THUMBV6-NEXT: sbcs r7, r2
-; THUMBV6-NEXT: ands r7, r3
-; THUMBV6-NEXT: orrs r7, r1
+; THUMBV6-NEXT: ldr r2, [sp, #4] @ 4-byte Reload
+; THUMBV6-NEXT: orrs r4, r2
+; THUMBV6-NEXT: subs r2, r4, #1
+; THUMBV6-NEXT: sbcs r4, r2
+; THUMBV6-NEXT: ands r4, r3
+; THUMBV6-NEXT: orrs r4, r1
+; THUMBV6-NEXT: ldr r1, [sp, #8] @ 4-byte Reload
+; THUMBV6-NEXT: orrs r4, r1
+; THUMBV6-NEXT: str r4, [sp, #56] @ 4-byte Spill
; THUMBV6-NEXT: ldr r1, [sp, #16] @ 4-byte Reload
-; THUMBV6-NEXT: orrs r7, r1
-; THUMBV6-NEXT: ldr r1, [sp, #24] @ 4-byte Reload
; THUMBV6-NEXT: adds r0, r0, r1
; THUMBV6-NEXT: str r0, [sp, #40] @ 4-byte Spill
-; THUMBV6-NEXT: ldr r0, [sp, #20] @ 4-byte Reload
-; THUMBV6-NEXT: adcs r4, r0
-; THUMBV6-NEXT: str r4, [sp, #32] @ 4-byte Spill
+; THUMBV6-NEXT: ldr r0, [sp, #12] @ 4-byte Reload
+; THUMBV6-NEXT: ldr r1, [sp, #52] @ 4-byte Reload
+; THUMBV6-NEXT: adcs r1, r0
+; THUMBV6-NEXT: str r1, [sp, #52] @ 4-byte Spill
; THUMBV6-NEXT: ldr r0, [sp, #44] @ 4-byte Reload
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: ldr r2, [sp, #56] @ 4-byte Reload
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: ldr r2, [sp, #32] @ 4-byte Reload
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
; THUMBV6-NEXT: mov r4, r1
-; THUMBV6-NEXT: ldr r1, [sp, #28] @ 4-byte Reload
-; THUMBV6-NEXT: adds r6, r0, r1
-; THUMBV6-NEXT: adcs r4, r5
+; THUMBV6-NEXT: ldr r1, [sp, #20] @ 4-byte Reload
+; THUMBV6-NEXT: adds r5, r0, r1
+; THUMBV6-NEXT: adcs r4, r6
; THUMBV6-NEXT: ldr r0, [sp, #36] @ 4-byte Reload
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: ldr r2, [sp, #52] @ 4-byte Reload
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: ldr r7, [sp, #24] @ 4-byte Reload
+; THUMBV6-NEXT: mov r2, r7
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
-; THUMBV6-NEXT: adds r0, r0, r6
+; THUMBV6-NEXT: adds r0, r0, r5
; THUMBV6-NEXT: ldr r2, [sp, #48] @ 4-byte Reload
; THUMBV6-NEXT: str r0, [r2, #4]
-; THUMBV6-NEXT: adcs r1, r5
+; THUMBV6-NEXT: adcs r1, r6
; THUMBV6-NEXT: adds r4, r4, r1
-; THUMBV6-NEXT: mov r6, r5
-; THUMBV6-NEXT: adcs r6, r5
+; THUMBV6-NEXT: mov r5, r6
+; THUMBV6-NEXT: adcs r5, r6
; THUMBV6-NEXT: ldr r0, [sp, #44] @ 4-byte Reload
-; THUMBV6-NEXT: mov r1, r5
-; THUMBV6-NEXT: ldr r2, [sp, #52] @ 4-byte Reload
-; THUMBV6-NEXT: mov r3, r5
+; THUMBV6-NEXT: mov r1, r6
+; THUMBV6-NEXT: mov r2, r7
+; THUMBV6-NEXT: mov r3, r6
; THUMBV6-NEXT: bl __aeabi_lmul
; THUMBV6-NEXT: adds r0, r0, r4
-; THUMBV6-NEXT: adcs r1, r6
+; THUMBV6-NEXT: adcs r1, r5
; THUMBV6-NEXT: ldr r2, [sp, #40] @ 4-byte Reload
; THUMBV6-NEXT: adds r0, r0, r2
-; THUMBV6-NEXT: ldr r2, [sp, #32] @ 4-byte Reload
+; THUMBV6-NEXT: ldr r2, [sp, #52] @ 4-byte Reload
; THUMBV6-NEXT: adcs r1, r2
; THUMBV6-NEXT: ldr r2, [sp, #48] @ 4-byte Reload
; THUMBV6-NEXT: str r0, [r2, #8]
; THUMBV6-NEXT: str r1, [r2, #12]
-; THUMBV6-NEXT: adcs r5, r5
-; THUMBV6-NEXT: orrs r5, r7
+; THUMBV6-NEXT: adcs r6, r6
+; THUMBV6-NEXT: ldr r0, [sp, #56] @ 4-byte Reload
+; THUMBV6-NEXT: orrs r6, r0
; THUMBV6-NEXT: movs r0, #1
-; THUMBV6-NEXT: ands r0, r5
+; THUMBV6-NEXT: ands r0, r6
; THUMBV6-NEXT: strb r0, [r2, #16]
; THUMBV6-NEXT: add sp, #60
; THUMBV6-NEXT: pop {r4, r5, r6, r7, pc}
diff --git a/llvm/test/CodeGen/Thumb2/umulo-128-legalisation-lowering.ll b/llvm/test/CodeGen/Thumb2/umulo-128-legalisation-lowering.ll
index fe1d06cb39e16..fcd4633fc7733 100644
--- a/llvm/test/CodeGen/Thumb2/umulo-128-legalisation-lowering.ll
+++ b/llvm/test/CodeGen/Thumb2/umulo-128-legalisation-lowering.ll
@@ -8,119 +8,117 @@ define { i128, i8 } @muloti_test(i128 %l, i128 %r) unnamed_addr #0 {
; THUMBV7-NEXT: push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
; THUMBV7-NEXT: .pad #44
; THUMBV7-NEXT: sub sp, #44
-; THUMBV7-NEXT: ldr.w r8, [sp, #88]
-; THUMBV7-NEXT: mov r9, r0
-; THUMBV7-NEXT: ldr r7, [sp, #96]
-; THUMBV7-NEXT: ldr.w lr, [sp, #100]
-; THUMBV7-NEXT: umull r0, r5, r2, r8
-; THUMBV7-NEXT: ldr r4, [sp, #80]
+; THUMBV7-NEXT: ldr r6, [sp, #84]
+; THUMBV7-NEXT: mov r8, r0
+; THUMBV7-NEXT: ldr r7, [sp, #92]
+; THUMBV7-NEXT: mov lr, r3
+; THUMBV7-NEXT: ldr r4, [sp, #96]
+; THUMBV7-NEXT: umull r0, r5, r1, r6
+; THUMBV7-NEXT: ldr.w r9, [sp, #88]
; THUMBV7-NEXT: str r0, [sp, #32] @ 4-byte Spill
-; THUMBV7-NEXT: umull r1, r0, r3, r7
+; THUMBV7-NEXT: umull r3, r0, r2, r7
; THUMBV7-NEXT: str r0, [sp, #4] @ 4-byte Spill
-; THUMBV7-NEXT: umull r0, r11, lr, r2
-; THUMBV7-NEXT: str r1, [sp, #20] @ 4-byte Spill
-; THUMBV7-NEXT: ldr r1, [sp, #92]
+; THUMBV7-NEXT: umull r0, r10, r4, r1
+; THUMBV7-NEXT: str r3, [sp, #20] @ 4-byte Spill
; THUMBV7-NEXT: str r0, [sp] @ 4-byte Spill
-; THUMBV7-NEXT: umull r0, r10, r7, r2
-; THUMBV7-NEXT: mov r7, r1
-; THUMBV7-NEXT: umull r6, r12, r1, r4
+; THUMBV7-NEXT: umull r0, r11, r7, r1
+; THUMBV7-NEXT: umull r7, r12, r9, lr
; THUMBV7-NEXT: str r0, [sp, #40] @ 4-byte Spill
-; THUMBV7-NEXT: ldr r0, [sp, #84]
-; THUMBV7-NEXT: str r6, [sp, #24] @ 4-byte Spill
-; THUMBV7-NEXT: umull r6, r1, r0, r8
-; THUMBV7-NEXT: str r6, [sp, #16] @ 4-byte Spill
-; THUMBV7-NEXT: umull r6, r2, r2, r7
-; THUMBV7-NEXT: mov r7, r4
-; THUMBV7-NEXT: strd r6, r2, [sp, #8] @ 8-byte Folded Spill
-; THUMBV7-NEXT: umull r2, r6, r4, r8
-; THUMBV7-NEXT: str r2, [sp, #36] @ 4-byte Spill
-; THUMBV7-NEXT: ldr r2, [sp, #32] @ 4-byte Reload
-; THUMBV7-NEXT: str r6, [sp, #28] @ 4-byte Spill
-; THUMBV7-NEXT: movs r6, #0
-; THUMBV7-NEXT: str.w r2, [r9]
-; THUMBV7-NEXT: umlal r5, r6, r3, r8
-; THUMBV7-NEXT: ldr r2, [sp, #20] @ 4-byte Reload
-; THUMBV7-NEXT: ldr r4, [sp] @ 4-byte Reload
-; THUMBV7-NEXT: add r4, r2
-; THUMBV7-NEXT: adds.w r2, r10, r4
-; THUMBV7-NEXT: str r2, [sp, #20] @ 4-byte Spill
-; THUMBV7-NEXT: mov.w r2, #0
-; THUMBV7-NEXT: adc r2, r2, #0
+; THUMBV7-NEXT: ldr r0, [sp, #80]
+; THUMBV7-NEXT: str r7, [sp, #24] @ 4-byte Spill
+; THUMBV7-NEXT: umull r7, r3, r0, r6
+; THUMBV7-NEXT: str r7, [sp, #16] @ 4-byte Spill
+; THUMBV7-NEXT: umull r7, r1, r1, r9
+; THUMBV7-NEXT: mov.w r9, #0
+; THUMBV7-NEXT: strd r7, r1, [sp, #8] @ 8-byte Folded Spill
+; THUMBV7-NEXT: umull r1, r7, lr, r6
+; THUMBV7-NEXT: str r1, [sp, #36] @ 4-byte Spill
+; THUMBV7-NEXT: ldr r1, [sp, #32] @ 4-byte Reload
+; THUMBV7-NEXT: str r7, [sp, #28] @ 4-byte Spill
+; THUMBV7-NEXT: movs r7, #0
+; THUMBV7-NEXT: str.w r1, [r8]
+; THUMBV7-NEXT: umlal r5, r7, r2, r6
+; THUMBV7-NEXT: ldr r1, [sp, #20] @ 4-byte Reload
+; THUMBV7-NEXT: ldr r6, [sp] @ 4-byte Reload
+; THUMBV7-NEXT: add r6, r1
+; THUMBV7-NEXT: adds.w r1, r11, r6
+; THUMBV7-NEXT: str r1, [sp, #20] @ 4-byte Spill
+; THUMBV7-NEXT: adc r1, r9, #0
; THUMBV7-NEXT: cmp.w r12, #0
-; THUMBV7-NEXT: str r2, [sp, #32] @ 4-byte Spill
+; THUMBV7-NEXT: str r1, [sp, #32] @ 4-byte Spill
; THUMBV7-NEXT: it ne
; THUMBV7-NEXT: movne.w r12, #1
-; THUMBV7-NEXT: cmp r1, #0
-; THUMBV7-NEXT: ldr r2, [sp, #96]
+; THUMBV7-NEXT: cmp r3, #0
+; THUMBV7-NEXT: ldr r1, [sp, #92]
; THUMBV7-NEXT: it ne
-; THUMBV7-NEXT: movne r1, #1
-; THUMBV7-NEXT: orrs.w r10, r7, r0
+; THUMBV7-NEXT: movne r3, #1
+; THUMBV7-NEXT: orrs.w lr, lr, r0
; THUMBV7-NEXT: it ne
-; THUMBV7-NEXT: movne.w r10, #1
-; THUMBV7-NEXT: orrs.w r7, r2, lr
-; THUMBV7-NEXT: ldr r2, [sp, #92]
+; THUMBV7-NEXT: movne.w lr, #1
+; THUMBV7-NEXT: orrs.w r9, r1, r4
+; THUMBV7-NEXT: ldr r1, [sp, #88]
; THUMBV7-NEXT: it ne
-; THUMBV7-NEXT: movne r7, #1
+; THUMBV7-NEXT: movne.w r9, #1
; THUMBV7-NEXT: cmp r0, #0
; THUMBV7-NEXT: it ne
; THUMBV7-NEXT: movne r0, #1
-; THUMBV7-NEXT: cmp r2, #0
-; THUMBV7-NEXT: mov r4, r2
-; THUMBV7-NEXT: mov r8, r2
+; THUMBV7-NEXT: cmp r1, #0
+; THUMBV7-NEXT: and.w lr, lr, r9
+; THUMBV7-NEXT: mov r6, r1
+; THUMBV7-NEXT: mov r11, r1
; THUMBV7-NEXT: it ne
-; THUMBV7-NEXT: movne r4, #1
-; THUMBV7-NEXT: ldr r2, [sp, #8] @ 4-byte Reload
-; THUMBV7-NEXT: ands r0, r4
-; THUMBV7-NEXT: movs r4, #0
-; THUMBV7-NEXT: adds r5, r5, r2
-; THUMBV7-NEXT: str.w r5, [r9, #4]
-; THUMBV7-NEXT: orr.w r0, r0, r1
+; THUMBV7-NEXT: movne r6, #1
+; THUMBV7-NEXT: ldr r1, [sp, #8] @ 4-byte Reload
+; THUMBV7-NEXT: ands r0, r6
+; THUMBV7-NEXT: adds r5, r5, r1
+; THUMBV7-NEXT: str.w r5, [r8, #4]
+; THUMBV7-NEXT: orr.w r0, r0, r3
; THUMBV7-NEXT: ldr r1, [sp, #24] @ 4-byte Reload
-; THUMBV7-NEXT: ldr r2, [sp, #16] @ 4-byte Reload
-; THUMBV7-NEXT: and.w r5, r10, r7
+; THUMBV7-NEXT: ldr r3, [sp, #16] @ 4-byte Reload
+; THUMBV7-NEXT: mov.w r5, #0
; THUMBV7-NEXT: orr.w r0, r0, r12
; THUMBV7-NEXT: mov.w r12, #0
-; THUMBV7-NEXT: add r1, r2
-; THUMBV7-NEXT: ldr r2, [sp, #12] @ 4-byte Reload
-; THUMBV7-NEXT: adcs r2, r6
-; THUMBV7-NEXT: ldr r6, [sp, #28] @ 4-byte Reload
-; THUMBV7-NEXT: adc r7, r4, #0
-; THUMBV7-NEXT: adds r1, r1, r6
-; THUMBV7-NEXT: umlal r2, r7, r3, r8
-; THUMBV7-NEXT: adc r4, r4, #0
-; THUMBV7-NEXT: orrs r0, r4
+; THUMBV7-NEXT: add r3, r1
+; THUMBV7-NEXT: ldr r1, [sp, #12] @ 4-byte Reload
+; THUMBV7-NEXT: adcs r1, r7
+; THUMBV7-NEXT: ldr r7, [sp, #28] @ 4-byte Reload
+; THUMBV7-NEXT: adc r6, r5, #0
+; THUMBV7-NEXT: adds r3, r3, r7
+; THUMBV7-NEXT: umlal r1, r6, r2, r11
+; THUMBV7-NEXT: adc r5, r5, #0
; THUMBV7-NEXT: orrs r0, r5
-; THUMBV7-NEXT: ldrd r5, r4, [sp, #36] @ 8-byte Folded Reload
-; THUMBV7-NEXT: adds r5, r5, r4
-; THUMBV7-NEXT: ldr r4, [sp, #20] @ 4-byte Reload
-; THUMBV7-NEXT: adcs r1, r4
-; THUMBV7-NEXT: ldr r4, [sp, #4] @ 4-byte Reload
-; THUMBV7-NEXT: cmp r4, #0
+; THUMBV7-NEXT: orr.w lr, lr, r0
+; THUMBV7-NEXT: ldrd r0, r5, [sp, #36] @ 8-byte Folded Reload
+; THUMBV7-NEXT: adds r5, r5, r0
+; THUMBV7-NEXT: ldr r0, [sp, #20] @ 4-byte Reload
+; THUMBV7-NEXT: adcs r3, r0
+; THUMBV7-NEXT: ldr r0, [sp, #4] @ 4-byte Reload
+; THUMBV7-NEXT: cmp r0, #0
; THUMBV7-NEXT: it ne
-; THUMBV7-NEXT: movne r4, #1
-; THUMBV7-NEXT: cmp r3, #0
+; THUMBV7-NEXT: movne r0, #1
+; THUMBV7-NEXT: cmp r2, #0
; THUMBV7-NEXT: it ne
-; THUMBV7-NEXT: movne r3, #1
-; THUMBV7-NEXT: cmp.w lr, #0
+; THUMBV7-NEXT: movne r2, #1
+; THUMBV7-NEXT: cmp r4, #0
; THUMBV7-NEXT: it ne
-; THUMBV7-NEXT: movne.w lr, #1
-; THUMBV7-NEXT: cmp.w r11, #0
+; THUMBV7-NEXT: movne r4, #1
+; THUMBV7-NEXT: cmp.w r10, #0
; THUMBV7-NEXT: it ne
-; THUMBV7-NEXT: movne.w r11, #1
-; THUMBV7-NEXT: adds r2, r2, r5
-; THUMBV7-NEXT: and.w r3, r3, lr
-; THUMBV7-NEXT: str.w r2, [r9, #8]
-; THUMBV7-NEXT: adcs r1, r7
-; THUMBV7-NEXT: str.w r1, [r9, #12]
-; THUMBV7-NEXT: orr.w r1, r3, r11
-; THUMBV7-NEXT: ldr r2, [sp, #32] @ 4-byte Reload
-; THUMBV7-NEXT: orr.w r1, r1, r4
-; THUMBV7-NEXT: orr.w r1, r1, r2
-; THUMBV7-NEXT: orr.w r0, r0, r1
+; THUMBV7-NEXT: movne.w r10, #1
+; THUMBV7-NEXT: adds r1, r1, r5
+; THUMBV7-NEXT: str.w r1, [r8, #8]
+; THUMBV7-NEXT: adcs.w r1, r6, r3
+; THUMBV7-NEXT: and.w r2, r2, r4
+; THUMBV7-NEXT: str.w r1, [r8, #12]
+; THUMBV7-NEXT: orr.w r1, r2, r10
+; THUMBV7-NEXT: orr.w r1, r1, r0
+; THUMBV7-NEXT: ldr r0, [sp, #32] @ 4-byte Reload
+; THUMBV7-NEXT: orr.w r1, r1, r0
+; THUMBV7-NEXT: orr.w r0, lr, r1
; THUMBV7-NEXT: adc r1, r12, #0
; THUMBV7-NEXT: orrs r0, r1
; THUMBV7-NEXT: and r0, r0, #1
-; THUMBV7-NEXT: strb.w r0, [r9, #16]
+; THUMBV7-NEXT: strb.w r0, [r8, #16]
; THUMBV7-NEXT: add sp, #44
; THUMBV7-NEXT: pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
start:
diff --git a/llvm/test/Instrumentation/BoundsChecking/runtimes.ll b/llvm/test/Instrumentation/BoundsChecking/runtimes.ll
index 2a756197dfd7e..1a0796a4344d9 100644
--- a/llvm/test/Instrumentation/BoundsChecking/runtimes.ll
+++ b/llvm/test/Instrumentation/BoundsChecking/runtimes.ll
@@ -21,7 +21,7 @@ define void @f1(i64 %x) nounwind {
; TR-LABEL: define void @f1(
; TR-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
; TR-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; TR-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; TR-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; TR-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
; TR-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
; TR-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
@@ -37,7 +37,7 @@ define void @f1(i64 %x) nounwind {
; RT-LABEL: define void @f1(
; RT-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
; RT-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; RT-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; RT-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; RT-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
; RT-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
; RT-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
@@ -53,7 +53,7 @@ define void @f1(i64 %x) nounwind {
; TR-NOMERGE-LABEL: define void @f1(
; TR-NOMERGE-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
; TR-NOMERGE-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; TR-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; TR-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; TR-NOMERGE-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
; TR-NOMERGE-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
; TR-NOMERGE-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
@@ -69,7 +69,7 @@ define void @f1(i64 %x) nounwind {
; RT-NOMERGE-LABEL: define void @f1(
; RT-NOMERGE-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
; RT-NOMERGE-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; RT-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; RT-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; RT-NOMERGE-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
; RT-NOMERGE-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
; RT-NOMERGE-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
@@ -85,7 +85,7 @@ define void @f1(i64 %x) nounwind {
; RTABORT-NOMERGE-LABEL: define void @f1(
; RTABORT-NOMERGE-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
; RTABORT-NOMERGE-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; RTABORT-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; RTABORT-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; RTABORT-NOMERGE-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
; RTABORT-NOMERGE-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
; RTABORT-NOMERGE-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
@@ -98,26 +98,10 @@ define void @f1(i64 %x) nounwind {
; RTABORT-NOMERGE-NEXT: call void @__ubsan_handle_local_out_of_bounds_abort() #[[ATTR2:[0-9]+]], !nosanitize [[META0]]
; RTABORT-NOMERGE-NEXT: unreachable, !nosanitize [[META0]]
;
-; MINRT-PRESERVE-NOMERGE-LABEL: define void @f1(
-; MINRT-PRESERVE-NOMERGE-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
-; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
-; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
-; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
-; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
-; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP6:%.*]] = or i1 false, [[TMP5]], !nosanitize [[META0]]
-; MINRT-PRESERVE-NOMERGE-NEXT: br i1 [[TMP6]], label %[[TRAP:.*]], label %[[BB7:.*]]
-; MINRT-PRESERVE-NOMERGE: [[BB7]]:
-; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP8:%.*]] = load i128, ptr [[TMP2]], align 4
-; MINRT-PRESERVE-NOMERGE-NEXT: ret void
-; MINRT-PRESERVE-NOMERGE: [[TRAP]]:
-; MINRT-PRESERVE-NOMERGE-NEXT: call preserve_allcc void @__ubsan_handle_local_out_of_bounds_minimal_preserve() #[[ATTR1:[0-9]+]], !nosanitize [[META0]]
-; MINRT-PRESERVE-NOMERGE-NEXT: br label %[[BB7]], !nosanitize [[META0]]
-;
; MINRT-NOMERGE-LABEL: define void @f1(
; MINRT-NOMERGE-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
; MINRT-NOMERGE-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; MINRT-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; MINRT-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; MINRT-NOMERGE-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
; MINRT-NOMERGE-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
; MINRT-NOMERGE-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
@@ -133,7 +117,7 @@ define void @f1(i64 %x) nounwind {
; MINRTABORT-NOMERGE-LABEL: define void @f1(
; MINRTABORT-NOMERGE-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
; MINRTABORT-NOMERGE-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; MINRTABORT-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; MINRTABORT-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; MINRTABORT-NOMERGE-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
; MINRTABORT-NOMERGE-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
; MINRTABORT-NOMERGE-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
@@ -146,34 +130,62 @@ define void @f1(i64 %x) nounwind {
; MINRTABORT-NOMERGE-NEXT: call void @__ubsan_handle_local_out_of_bounds_minimal_abort() #[[ATTR2:[0-9]+]], !nosanitize [[META0]]
; MINRTABORT-NOMERGE-NEXT: unreachable, !nosanitize [[META0]]
;
-; TR-GUARD-COMMON-LABEL: define void @f1(
-; TR-GUARD-COMMON-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
-; TR-GUARD-COMMON-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; TR-GUARD-COMMON-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
-; TR-GUARD-COMMON-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
-; TR-GUARD-COMMON-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
-; TR-GUARD-COMMON-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
-; TR-GUARD-COMMON-NEXT: [[TMP6:%.*]] = or i1 false, [[TMP5]], !nosanitize [[META0]]
-;
-; TR-GUARD-THREE: [[TMP7:%.*]] = call i1 @llvm.allow.ubsan.check(i8 3), !nosanitize [[META0]]
-; TR-GUARD-THIRTEEN: [[TMP7:%.*]] = call i1 @llvm.allow.ubsan.check(i8 13), !nosanitize [[META0]]
-;
-; TR-GUARD-COMMON: [[TMP8:%.*]] = and i1 [[TMP6]], [[TMP7]], !nosanitize [[META0]]
-; TR-GUARD-COMMON-NEXT: br i1 [[TMP8]], label %[[TRAP:.*]], label %[[BB9:.*]]
-; TR-GUARD-COMMON: [[BB9]]:
-; TR-GUARD-COMMON-NEXT: [[TMP10:%.*]] = load i128, ptr [[TMP2]], align 4
-; TR-GUARD-COMMON-NEXT: ret void
-; TR-GUARD-COMMON: [[TRAP]]:
+; MINRT-PRESERVE-NOMERGE-LABEL: define void @f1(
+; MINRT-PRESERVE-NOMERGE-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
+; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
+; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
+; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
+; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
+; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
+; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP6:%.*]] = or i1 false, [[TMP5]], !nosanitize [[META0]]
+; MINRT-PRESERVE-NOMERGE-NEXT: br i1 [[TMP6]], label %[[TRAP:.*]], label %[[BB7:.*]]
+; MINRT-PRESERVE-NOMERGE: [[BB7]]:
+; MINRT-PRESERVE-NOMERGE-NEXT: [[TMP8:%.*]] = load i128, ptr [[TMP2]], align 4
+; MINRT-PRESERVE-NOMERGE-NEXT: ret void
+; MINRT-PRESERVE-NOMERGE: [[TRAP]]:
+; MINRT-PRESERVE-NOMERGE-NEXT: call preserve_allcc void @__ubsan_handle_local_out_of_bounds_minimal_preserve() #[[ATTR1:[0-9]+]], !nosanitize [[META0]]
+; MINRT-PRESERVE-NOMERGE-NEXT: br label %[[BB7]], !nosanitize [[META0]]
;
-; TR-GUARD-THREE: call void @llvm.ubsantrap(i8 3) #[[ATTR3:[0-9]+]], !nosanitize [[META0]]
-; TR-GUARD-THIRTEEN: call void @llvm.ubsantrap(i8 13) #[[ATTR3:[0-9]+]], !nosanitize [[META0]]
+; TR-GUARD-THREE-LABEL: define void @f1(
+; TR-GUARD-THREE-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
+; TR-GUARD-THREE-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
+; TR-GUARD-THREE-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
+; TR-GUARD-THREE-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
+; TR-GUARD-THREE-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
+; TR-GUARD-THREE-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
+; TR-GUARD-THREE-NEXT: [[TMP6:%.*]] = or i1 false, [[TMP5]], !nosanitize [[META0]]
+; TR-GUARD-THREE-NEXT: [[TMP7:%.*]] = call i1 @llvm.allow.ubsan.check(i8 3), !nosanitize [[META0]]
+; TR-GUARD-THREE-NEXT: [[TMP8:%.*]] = and i1 [[TMP6]], [[TMP7]], !nosanitize [[META0]]
+; TR-GUARD-THREE-NEXT: br i1 [[TMP8]], label %[[TRAP:.*]], label %[[BB9:.*]]
+; TR-GUARD-THREE: [[BB9]]:
+; TR-GUARD-THREE-NEXT: [[TMP10:%.*]] = load i128, ptr [[TMP2]], align 4
+; TR-GUARD-THREE-NEXT: ret void
+; TR-GUARD-THREE: [[TRAP]]:
+; TR-GUARD-THREE-NEXT: call void @llvm.ubsantrap(i8 3) #[[ATTR3:[0-9]+]], !nosanitize [[META0]]
+; TR-GUARD-THREE-NEXT: unreachable, !nosanitize [[META0]]
;
-; TR-GUARD-COMMON: unreachable, !nosanitize [[META0]]
+; TR-GUARD-THIRTEEN-LABEL: define void @f1(
+; TR-GUARD-THIRTEEN-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
+; TR-GUARD-THIRTEEN-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
+; TR-GUARD-THIRTEEN-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
+; TR-GUARD-THIRTEEN-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
+; TR-GUARD-THIRTEEN-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
+; TR-GUARD-THIRTEEN-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
+; TR-GUARD-THIRTEEN-NEXT: [[TMP6:%.*]] = or i1 false, [[TMP5]], !nosanitize [[META0]]
+; TR-GUARD-THIRTEEN-NEXT: [[TMP7:%.*]] = call i1 @llvm.allow.ubsan.check(i8 13), !nosanitize [[META0]]
+; TR-GUARD-THIRTEEN-NEXT: [[TMP8:%.*]] = and i1 [[TMP6]], [[TMP7]], !nosanitize [[META0]]
+; TR-GUARD-THIRTEEN-NEXT: br i1 [[TMP8]], label %[[TRAP:.*]], label %[[BB9:.*]]
+; TR-GUARD-THIRTEEN: [[BB9]]:
+; TR-GUARD-THIRTEEN-NEXT: [[TMP10:%.*]] = load i128, ptr [[TMP2]], align 4
+; TR-GUARD-THIRTEEN-NEXT: ret void
+; TR-GUARD-THIRTEEN: [[TRAP]]:
+; TR-GUARD-THIRTEEN-NEXT: call void @llvm.ubsantrap(i8 13) #[[ATTR3:[0-9]+]], !nosanitize [[META0]]
+; TR-GUARD-THIRTEEN-NEXT: unreachable, !nosanitize [[META0]]
;
; RT-GUARD-LABEL: define void @f1(
; RT-GUARD-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
; RT-GUARD-NEXT: [[TMP1:%.*]] = mul i64 [[X]], 16
-; RT-GUARD-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; RT-GUARD-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; RT-GUARD-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0:![0-9]+]]
; RT-GUARD-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
; RT-GUARD-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
@@ -193,6 +205,11 @@ define void @f1(i64 %x) nounwind {
ret void
}
+; TR-GUARD: attributes #[[ATTR0]] = { nounwind }
+; TR-GUARD: attributes #[[ATTR1:[0-9]+]] = { nocallback nofree nosync nounwind willreturn memory(inaccessiblemem: write) }
+; TR-GUARD: attributes #[[ATTR2:[0-9]+]] = { cold noreturn nounwind memory(inaccessiblemem: write) }
+; TR-GUARD: attributes #[[ATTR3]] = { nomerge noreturn nounwind }
+; TR-GUARD: [[META0]] = !{}
;.
; TR: attributes #[[ATTR0]] = { nounwind }
; TR: attributes #[[ATTR1:[0-9]+]] = { cold noreturn nounwind memory(inaccessiblemem: write) }
@@ -218,10 +235,18 @@ define void @f1(i64 %x) nounwind {
; MINRTABORT-NOMERGE: attributes #[[ATTR1:[0-9]+]] = { noreturn nounwind }
; MINRTABORT-NOMERGE: attributes #[[ATTR2]] = { nomerge noreturn nounwind }
;.
-; TR-GUARD: attributes #[[ATTR0]] = { nounwind }
-; TR-GUARD: attributes #[[ATTR1:[0-9]+]] = { nocallback nofree nosync nounwind willreturn memory(inaccessiblemem: write) }
-; TR-GUARD: attributes #[[ATTR2:[0-9]+]] = { cold noreturn nounwind memory(inaccessiblemem: write) }
-; TR-GUARD: attributes #[[ATTR3]] = { nomerge noreturn nounwind }
+; MINRT-PRESERVE-NOMERGE: attributes #[[ATTR0]] = { nounwind }
+; MINRT-PRESERVE-NOMERGE: attributes #[[ATTR1]] = { nomerge nounwind }
+;.
+; TR-GUARD-THREE: attributes #[[ATTR0]] = { nounwind }
+; TR-GUARD-THREE: attributes #[[ATTR1:[0-9]+]] = { nocallback nofree nosync nounwind willreturn memory(inaccessiblemem: readwrite) }
+; TR-GUARD-THREE: attributes #[[ATTR2:[0-9]+]] = { cold noreturn nounwind memory(inaccessiblemem: write) }
+; TR-GUARD-THREE: attributes #[[ATTR3]] = { nomerge noreturn nounwind }
+;.
+; TR-GUARD-THIRTEEN: attributes #[[ATTR0]] = { nounwind }
+; TR-GUARD-THIRTEEN: attributes #[[ATTR1:[0-9]+]] = { nocallback nofree nosync nounwind willreturn memory(inaccessiblemem: readwrite) }
+; TR-GUARD-THIRTEEN: attributes #[[ATTR2:[0-9]+]] = { cold noreturn nounwind memory(inaccessiblemem: write) }
+; TR-GUARD-THIRTEEN: attributes #[[ATTR3]] = { nomerge noreturn nounwind }
;.
; RT-GUARD: attributes #[[ATTR0]] = { nounwind }
; RT-GUARD: attributes #[[ATTR1:[0-9]+]] = { nocallback nofree nosync nounwind willreturn memory(inaccessiblemem: readwrite) }
@@ -241,7 +266,13 @@ define void @f1(i64 %x) nounwind {
;.
; MINRTABORT-NOMERGE: [[META0]] = !{}
;.
-; TR-GUARD: [[META0]] = !{}
+; MINRT-PRESERVE-NOMERGE: [[META0]] = !{}
+;.
+; TR-GUARD-THREE: [[META0]] = !{}
+;.
+; TR-GUARD-THIRTEEN: [[META0]] = !{}
;.
; RT-GUARD: [[META0]] = !{}
;.
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; TR-GUARD-COMMON: {{.*}}
diff --git a/llvm/test/Instrumentation/BoundsChecking/simple.ll b/llvm/test/Instrumentation/BoundsChecking/simple.ll
index 81592d79d3a83..90a531b8b2e0c 100644
--- a/llvm/test/Instrumentation/BoundsChecking/simple.ll
+++ b/llvm/test/Instrumentation/BoundsChecking/simple.ll
@@ -171,7 +171,7 @@ define void @f5_as2(i32 %x) nounwind {;
define void @f6(i64 %x) nounwind {
; CHECK-LABEL: @f6(
-; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 8
+; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 16
; CHECK-NEXT: [[TMP2:%.*]] = load i128, ptr [[TMP1]], align 4
; CHECK-NEXT: ret void
;
@@ -183,7 +183,7 @@ define void @f6(i64 %x) nounwind {
define void @f7(i64 %x) nounwind {
; CHECK-LABEL: @f7(
; CHECK-NEXT: [[TMP1:%.*]] = mul i64 [[X:%.*]], 16
-; CHECK-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; CHECK-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; CHECK-NEXT: [[TMP3:%.*]] = sub i64 [[TMP1]], 0, !nosanitize [[META0]]
; CHECK-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP3]], 16, !nosanitize [[META0]]
; CHECK-NEXT: [[TMP5:%.*]] = or i1 false, [[TMP4]], !nosanitize [[META0]]
@@ -203,8 +203,8 @@ define void @f7(i64 %x) nounwind {
define void @f8() nounwind {
; CHECK-LABEL: @f8(
-; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 8
-; CHECK-NEXT: [[TMP2:%.*]] = alloca i128, align 8
+; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 16
+; CHECK-NEXT: [[TMP2:%.*]] = alloca i128, align 16
; CHECK-NEXT: [[TMP3:%.*]] = select i1 undef, ptr [[TMP1]], ptr [[TMP2]]
; CHECK-NEXT: [[TMP4:%.*]] = load i128, ptr [[TMP3]], align 4
; CHECK-NEXT: ret void
@@ -218,7 +218,7 @@ define void @f8() nounwind {
define void @f9(ptr %arg) nounwind {
; CHECK-LABEL: @f9(
-; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 8
+; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 16
; CHECK-NEXT: [[TMP2:%.*]] = select i1 undef, ptr [[ARG:%.*]], ptr [[TMP1]]
; CHECK-NEXT: [[TMP3:%.*]] = load i128, ptr [[TMP2]], align 4
; CHECK-NEXT: ret void
@@ -232,9 +232,9 @@ define void @f9(ptr %arg) nounwind {
define void @f10(i64 %x, i64 %y) nounwind {
; CHECK-LABEL: @f10(
; CHECK-NEXT: [[TMP1:%.*]] = mul i64 [[X:%.*]], 16
-; CHECK-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 8
+; CHECK-NEXT: [[TMP2:%.*]] = alloca i128, i64 [[X]], align 16
; CHECK-NEXT: [[TMP3:%.*]] = mul i64 [[Y:%.*]], 16
-; CHECK-NEXT: [[TMP4:%.*]] = alloca i128, i64 [[Y]], align 8
+; CHECK-NEXT: [[TMP4:%.*]] = alloca i128, i64 [[Y]], align 16
; CHECK-NEXT: [[TMP5:%.*]] = select i1 undef, i64 [[TMP1]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP6:%.*]] = select i1 undef, ptr [[TMP2]], ptr [[TMP4]]
; CHECK-NEXT: [[TMP7:%.*]] = sub i64 [[TMP5]], 0, !nosanitize [[META0]]
@@ -532,14 +532,14 @@ define void @scalable_alloca2(i64 %y) nounwind {
; CHECK-NEXT: [[DOTIDX:%.*]] = mul i64 [[Y:%.*]], [[TMP6]]
; CHECK-NEXT: [[TMP7:%.*]] = add i64 0, [[DOTIDX]]
; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds <vscale x 4 x i64>, ptr [[TMP4]], i64 [[Y]]
-; CHECK-NEXT: [[TMP9:%.*]] = call i64 @llvm.vscale.i64(), !nosanitize [[META0]]
-; CHECK-NEXT: [[TMP10:%.*]] = mul nuw i64 [[TMP9]], 32, !nosanitize [[META0]]
-; CHECK-NEXT: [[TMP11:%.*]] = sub i64 [[TMP2]], [[TMP7]], !nosanitize [[META0]]
-; CHECK-NEXT: [[TMP12:%.*]] = icmp ult i64 [[TMP2]], [[TMP7]], !nosanitize [[META0]]
-; CHECK-NEXT: [[TMP13:%.*]] = icmp ult i64 [[TMP11]], [[TMP10]], !nosanitize [[META0]]
-; CHECK-NEXT: [[TMP14:%.*]] = or i1 [[TMP12]], [[TMP13]], !nosanitize [[META0]]
-; CHECK-NEXT: [[TMP15:%.*]] = icmp slt i64 [[TMP7]], 0, !nosanitize [[META0]]
-; CHECK-NEXT: [[TMP16:%.*]] = or i1 [[TMP15]], [[TMP14]], !nosanitize [[META0]]
+; CHECK-NEXT: [[TMP15:%.*]] = call i64 @llvm.vscale.i64(), !nosanitize [[META0]]
+; CHECK-NEXT: [[TMP9:%.*]] = mul nuw i64 [[TMP15]], 32, !nosanitize [[META0]]
+; CHECK-NEXT: [[TMP10:%.*]] = sub i64 [[TMP2]], [[TMP7]], !nosanitize [[META0]]
+; CHECK-NEXT: [[TMP11:%.*]] = icmp ult i64 [[TMP2]], [[TMP7]], !nosanitize [[META0]]
+; CHECK-NEXT: [[TMP12:%.*]] = icmp ult i64 [[TMP10]], [[TMP9]], !nosanitize [[META0]]
+; CHECK-NEXT: [[TMP13:%.*]] = or i1 [[TMP11]], [[TMP12]], !nosanitize [[META0]]
+; CHECK-NEXT: [[TMP14:%.*]] = icmp slt i64 [[TMP7]], 0, !nosanitize [[META0]]
+; CHECK-NEXT: [[TMP16:%.*]] = or i1 [[TMP14]], [[TMP13]], !nosanitize [[META0]]
; CHECK-NEXT: br i1 [[TMP16]], label [[TRAP:%.*]], label [[TMP17:%.*]]
; CHECK: 16:
; CHECK-NEXT: [[TMP18:%.*]] = load <vscale x 4 x i64>, ptr [[TMP8]], align 4
diff --git a/llvm/test/Instrumentation/DataFlowSanitizer/load.ll b/llvm/test/Instrumentation/DataFlowSanitizer/load.ll
index bf8ba909e0be0..0bac48675855b 100644
--- a/llvm/test/Instrumentation/DataFlowSanitizer/load.ll
+++ b/llvm/test/Instrumentation/DataFlowSanitizer/load.ll
@@ -114,7 +114,7 @@ define i128 @load128(ptr %p) {
; CHECK-NEXT: %[[#WIDE_SHADOW:]] = or i64 %[[#WIDE_SHADOW]], %[[#WIDE_SHADOW_SHIFTED]]
; CHECK-NEXT: %[[#SHADOW:]] = trunc i64 %[[#WIDE_SHADOW]] to i8
; COMBINE_LOAD_PTR-NEXT: %[[#SHADOW:]] = or i8 %[[#SHADOW]], %[[#PS]]
- ; CHECK-NEXT: %a = load i128, ptr %p, align 8
+ ; CHECK-NEXT: %a = load i128, ptr %p, align 16
; CHECK-NEXT: store i8 %[[#SHADOW]], ptr @__dfsan_retval_tls, align [[ALIGN]]
; CHECK-NEXT: ret i128 %a
diff --git a/llvm/test/Instrumentation/DataFlowSanitizer/origin_load.ll b/llvm/test/Instrumentation/DataFlowSanitizer/origin_load.ll
index a0c642a3cd0e1..bdda5fc4d275c 100644
--- a/llvm/test/Instrumentation/DataFlowSanitizer/origin_load.ll
+++ b/llvm/test/Instrumentation/DataFlowSanitizer/origin_load.ll
@@ -224,19 +224,19 @@ define i128 @load128(ptr %p) {
; CHECK-NEXT: %[[#SHADOW_PTR:]] = inttoptr i64 %[[#SHADOW_OFFSET]] to ptr
; CHECK-NEXT: %[[#ORIGIN_ADDR:]] = add i64 %[[#SHADOW_OFFSET]], [[#ORIGIN_BASE]]
; CHECK-NEXT: %[[#ORIGIN1_PTR:]] = inttoptr i64 %[[#ORIGIN_ADDR]] to ptr
- ; CHECK-NEXT: %[[#ORIGIN1:]] = load i32, ptr %[[#ORIGIN1_PTR]], align 8
+ ; CHECK-NEXT: %[[#ORIGIN1:]] = load i32, ptr %[[#ORIGIN1_PTR]], align 16
; CHECK-NEXT: %[[#WIDE_SHADOW1:]] = load i64, ptr %[[#SHADOW_PTR]], align 1
; CHECK-NEXT: %[[#WIDE_SHADOW1_LO:]] = shl i64 %[[#WIDE_SHADOW1]], 32
; CHECK-NEXT: %[[#ORIGIN2_PTR:]] = getelementptr i32, ptr %[[#ORIGIN1_PTR]], i64 1
- ; CHECK-NEXT: %[[#ORIGIN2:]] = load i32, ptr %[[#ORIGIN2_PTR]], align 8
+ ; CHECK-NEXT: %[[#ORIGIN2:]] = load i32, ptr %[[#ORIGIN2_PTR]], align 16
; CHECK-NEXT: %[[#WIDE_SHADOW2_PTR:]] = getelementptr i64, ptr %[[#SHADOW_PTR]], i64 1
; CHECK-NEXT: %[[#WIDE_SHADOW2:]] = load i64, ptr %[[#WIDE_SHADOW2_PTR]], align 1
; CHECK-NEXT: %[[#WIDE_SHADOW:]] = or i64 %[[#WIDE_SHADOW1]], %[[#WIDE_SHADOW2]]
; CHECK-NEXT: %[[#ORIGIN3_PTR:]] = getelementptr i32, ptr %[[#ORIGIN2_PTR]], i64 1
- ; CHECK-NEXT: %[[#ORIGIN3:]] = load i32, ptr %[[#ORIGIN3_PTR]], align 8
+ ; CHECK-NEXT: %[[#ORIGIN3:]] = load i32, ptr %[[#ORIGIN3_PTR]], align 16
; CHECK-NEXT: %[[#WIDE_SHADOW2_LO:]] = shl i64 %[[#WIDE_SHADOW2]], 32
; CHECK-NEXT: %[[#ORIGIN4_PTR:]] = getelementptr i32, ptr %[[#ORIGIN3_PTR]], i64 1
- ; CHECK-NEXT: %[[#ORIGIN4:]] = load i32, ptr %[[#ORIGIN4_PTR]], align 8
+ ; CHECK-NEXT: %[[#ORIGIN4:]] = load i32, ptr %[[#ORIGIN4_PTR]], align 16
; CHECK-NEXT: %[[#WIDE_SHADOW_SHIFTED:]] = lshr i64 %[[#WIDE_SHADOW]], 32
; CHECK-NEXT: %[[#WIDE_SHADOW:]] = or i64 %[[#WIDE_SHADOW]], %[[#WIDE_SHADOW_SHIFTED]]
; CHECK-NEXT: %[[#WIDE_SHADOW_SHIFTED:]] = lshr i64 %[[#WIDE_SHADOW]], 16
@@ -255,7 +255,7 @@ define i128 @load128(ptr %p) {
; COMBINE_LOAD_PTR-NEXT: %[[#NZ:]] = icmp ne i8 %[[#PS]], 0
; COMBINE_LOAD_PTR-NEXT: %[[#ORIGIN:]] = select i1 %[[#NZ]], i32 %[[#PO]], i32 %[[#ORIGIN]]
- ; CHECK-NEXT: %a = load i128, ptr %p, align 8
+ ; CHECK-NEXT: %a = load i128, ptr %p, align 16
; CHECK-NEXT: store i8 %[[#SHADOW]], ptr @__dfsan_retval_tls, align [[ALIGN]]
; CHECK-NEXT: store i32 %[[#ORIGIN]], ptr @__dfsan_retval_origin_tls, align 4
diff --git a/llvm/test/Instrumentation/Instrumentor/cast.ll b/llvm/test/Instrumentation/Instrumentor/cast.ll
index 77d9e0452784d..a1660dba489fa 100644
--- a/llvm/test/Instrumentation/Instrumentor/cast.ll
+++ b/llvm/test/Instrumentation/Instrumentor/cast.ll
@@ -275,15 +275,15 @@ entry:
define i128 @test_ext(i32 %p1) {
; CHECK-LABEL: define i128 @test_ext(
; CHECK-SAME: i32 [[P1:%.*]]) {
-; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 8
+; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 16
; CHECK-NEXT: [[TMP2:%.*]] = alloca i64, align 8
; CHECK-NEXT: [[TMP3:%.*]] = zext i32 [[P1]] to i64
; CHECK-NEXT: call void @__instrumentor_pre_cast(i64 [[TMP3]], i32 12, i32 4, i32 12, i32 16, i32 40) #[[ATTR0]]
; CHECK-NEXT: [[I1:%.*]] = zext i32 [[P1]] to i128
; CHECK-NEXT: store i64 [[TMP3]], ptr [[TMP2]], align 4
-; CHECK-NEXT: store i128 [[I1]], ptr [[TMP1]], align 4
+; CHECK-NEXT: store i128 [[I1]], ptr [[TMP1]], align 16
; CHECK-NEXT: call void @__instrumentor_post_cast_ind(ptr [[TMP2]], i32 12, i32 4, ptr [[TMP1]], i32 12, i32 16, i32 40) #[[ATTR0]]
-; CHECK-NEXT: [[TMP4:%.*]] = load i128, ptr [[TMP1]], align 4
+; CHECK-NEXT: [[TMP4:%.*]] = load i128, ptr [[TMP1]], align 16
; CHECK-NEXT: ret i128 [[TMP4]]
;
%i1 = zext i32 %p1 to i128
diff --git a/llvm/test/Instrumentation/Instrumentor/cast_crash.ll b/llvm/test/Instrumentation/Instrumentor/cast_crash.ll
index f9bf05c9c0cca..6067686424c88 100644
--- a/llvm/test/Instrumentation/Instrumentor/cast_crash.ll
+++ b/llvm/test/Instrumentation/Instrumentor/cast_crash.ll
@@ -5,9 +5,9 @@
define i128 @test_ext(i32 %p1) {
; CHECK-LABEL: define i128 @test_ext(
; CHECK-SAME: i32 [[P1:%.*]]) {
-; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 8
+; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 16
; CHECK-NEXT: [[TMP2:%.*]] = alloca i64, align 8
-; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr align 8 [[TMP1]], ptr @__instrumentor_value_pack, i64 16, i1 false)
+; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr align 16 [[TMP1]], ptr @__instrumentor_value_pack, i64 16, i1 false)
; CHECK-NEXT: [[TMP3:%.*]] = getelementptr inbounds nuw <{ i32, i32, [4 x i8], i32 }>, ptr [[TMP1]], i32 0, i32 3
; CHECK-NEXT: store i32 [[P1]], ptr [[TMP3]], align 4
; CHECK-NEXT: call void @__instrumentor_pre_function(ptr @test_ext, ptr @__instrumentor_.str.2, i32 1, ptr [[TMP1]], i8 0, i32 4) #[[ATTR1:[0-9]+]]
@@ -17,10 +17,10 @@ define i128 @test_ext(i32 %p1) {
; CHECK-NEXT: call void @__instrumentor_pre_cast(i64 [[TMP6]], i32 12, i32 -1, i32 4, i32 12, i32 -1, i32 16, i32 40, i32 3) #[[ATTR1]]
; CHECK-NEXT: [[I1:%.*]] = zext i32 [[TMP5]] to i128
; CHECK-NEXT: store i64 [[TMP6]], ptr [[TMP2]], align 4
-; CHECK-NEXT: store i128 [[I1]], ptr [[TMP1]], align 4
+; CHECK-NEXT: store i128 [[I1]], ptr [[TMP1]], align 16
; CHECK-NEXT: call void @__instrumentor_post_cast_ind(ptr [[TMP2]], i32 12, i32 -1, i32 4, ptr [[TMP1]], i32 12, i32 -1, i32 16, i32 40, i32 -3) #[[ATTR1]]
-; CHECK-NEXT: [[TMP7:%.*]] = load i128, ptr [[TMP1]], align 4
-; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr align 8 [[TMP1]], ptr @__instrumentor_value_pack, i64 16, i1 false)
+; CHECK-NEXT: [[TMP7:%.*]] = load i128, ptr [[TMP1]], align 16
+; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr align 16 [[TMP1]], ptr @__instrumentor_value_pack, i64 16, i1 false)
; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds nuw <{ i32, i32, [4 x i8], i32 }>, ptr [[TMP1]], i32 0, i32 3
; CHECK-NEXT: store i32 [[P1]], ptr [[TMP8]], align 4
; CHECK-NEXT: call void @__instrumentor_post_function(ptr @test_ext, ptr @__instrumentor_.str.2, i32 1, ptr [[TMP1]], i8 0, i32 -5) #[[ATTR1]]
diff --git a/llvm/test/Instrumentation/Instrumentor/load_store_gpu_ind.ll b/llvm/test/Instrumentation/Instrumentor/load_store_gpu_ind.ll
index dd88398964b83..781c5af7009b6 100644
--- a/llvm/test/Instrumentation/Instrumentor/load_store_gpu_ind.ll
+++ b/llvm/test/Instrumentation/Instrumentor/load_store_gpu_ind.ll
@@ -123,8 +123,8 @@ define noundef i128 @_Z20store_load_long_longPx(ptr captures(none) noundef initi
; CHECK-LABEL: define noundef i128 @_Z20store_load_long_longPx(
; CHECK-SAME: ptr noundef captures(none) initializes((0, 16)) [[A:%.*]]) {
; CHECK-NEXT: [[ENTRY:.*:]]
-; CHECK-NEXT: [[TMP0:%.*]] = alloca i128, align 8, addrspace(5)
-; CHECK-NEXT: store i128 5, ptr addrspace(5) [[TMP0]], align 8
+; CHECK-NEXT: [[TMP0:%.*]] = alloca i128, align 16, addrspace(5)
+; CHECK-NEXT: store i128 5, ptr addrspace(5) [[TMP0]], align 16
; CHECK-NEXT: [[TMP1:%.*]] = addrspacecast ptr addrspace(5) [[TMP0]] to ptr
; CHECK-NEXT: [[TMP2:%.*]] = call ptr @__instrumentor_pre_store_ind(ptr [[A]], i32 0, ptr [[TMP1]], i64 16, i64 8, i32 12, i32 0, i8 1, i8 0) #[[ATTR0]]
; CHECK-NEXT: store i128 5, ptr [[TMP2]], align 8
@@ -132,10 +132,10 @@ define noundef i128 @_Z20store_load_long_longPx(ptr captures(none) noundef initi
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds nuw i8, ptr [[A]], i64 16
; CHECK-NEXT: [[TMP3:%.*]] = call ptr @__instrumentor_pre_load(ptr [[ARRAYIDX]], i32 0, i64 16, i64 8, i32 12, i32 0, i8 1, i8 0) #[[ATTR0]]
; CHECK-NEXT: [[TMP6:%.*]] = load i128, ptr [[TMP3]], align 8
-; CHECK-NEXT: store i128 [[TMP6]], ptr addrspace(5) [[TMP0]], align 8
+; CHECK-NEXT: store i128 [[TMP6]], ptr addrspace(5) [[TMP0]], align 16
; CHECK-NEXT: [[TMP5:%.*]] = addrspacecast ptr addrspace(5) [[TMP0]] to ptr
; CHECK-NEXT: call void @__instrumentor_post_load_ind(ptr [[ARRAYIDX]], i32 0, ptr [[TMP5]], i64 16, i64 8, i32 12, i32 0, i8 1, i8 0) #[[ATTR0]]
-; CHECK-NEXT: [[TMP4:%.*]] = load i128, ptr [[TMP5]], align 8
+; CHECK-NEXT: [[TMP4:%.*]] = load i128, ptr [[TMP5]], align 16
; CHECK-NEXT: ret i128 [[TMP4]]
;
entry:
diff --git a/llvm/test/Instrumentation/Instrumentor/numeric.ll b/llvm/test/Instrumentation/Instrumentor/numeric.ll
index b101b0c860bb0..fcb7b1d107e24 100644
--- a/llvm/test/Instrumentation/Instrumentor/numeric.ll
+++ b/llvm/test/Instrumentation/Instrumentor/numeric.ll
@@ -228,44 +228,44 @@ define i128 @test_i128(i128 %p1, i128 %p2) {
; CHECK-LABEL: define i128 @test_i128(
; CHECK-SAME: i128 [[P1:%.*]], i128 [[P2:%.*]]) {
; CHECK-NEXT: [[ENTRY:.*:]]
-; CHECK-NEXT: [[TMP0:%.*]] = alloca i128, align 8
-; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 8
-; CHECK-NEXT: [[TMP2:%.*]] = alloca i128, align 8
-; CHECK-NEXT: store i128 [[P1]], ptr [[TMP2]], align 4
-; CHECK-NEXT: store i128 [[P2]], ptr [[TMP1]], align 4
+; CHECK-NEXT: [[TMP0:%.*]] = alloca i128, align 16
+; CHECK-NEXT: [[TMP1:%.*]] = alloca i128, align 16
+; CHECK-NEXT: [[TMP2:%.*]] = alloca i128, align 16
+; CHECK-NEXT: store i128 [[P1]], ptr [[TMP2]], align 16
+; CHECK-NEXT: store i128 [[P2]], ptr [[TMP1]], align 16
; CHECK-NEXT: call void @__instrumentor_pre_numeric_ind(i32 12, i32 16, i32 14, ptr [[TMP2]], ptr [[TMP1]], i64 0, i32 26) #[[ATTR0]]
; CHECK-NEXT: [[A1:%.*]] = add i128 [[P1]], [[P2]]
-; CHECK-NEXT: store i128 [[A1]], ptr [[TMP0]], align 4
+; CHECK-NEXT: store i128 [[A1]], ptr [[TMP0]], align 16
; CHECK-NEXT: call void @__instrumentor_post_numeric_ind(i32 12, i32 16, i32 14, ptr [[TMP2]], ptr [[TMP1]], ptr [[TMP0]], i64 0, i32 -26) #[[ATTR0]]
-; CHECK-NEXT: store i128 [[P1]], ptr [[TMP2]], align 4
-; CHECK-NEXT: store i128 [[A1]], ptr [[TMP0]], align 4
+; CHECK-NEXT: store i128 [[P1]], ptr [[TMP2]], align 16
+; CHECK-NEXT: store i128 [[A1]], ptr [[TMP0]], align 16
; CHECK-NEXT: call void @__instrumentor_pre_numeric_ind(i32 12, i32 16, i32 18, ptr [[TMP2]], ptr [[TMP0]], i64 0, i32 27) #[[ATTR0]]
; CHECK-NEXT: [[A2:%.*]] = mul i128 [[P1]], [[A1]]
-; CHECK-NEXT: store i128 [[A2]], ptr [[TMP1]], align 4
+; CHECK-NEXT: store i128 [[A2]], ptr [[TMP1]], align 16
; CHECK-NEXT: call void @__instrumentor_post_numeric_ind(i32 12, i32 16, i32 18, ptr [[TMP2]], ptr [[TMP0]], ptr [[TMP1]], i64 0, i32 -27) #[[ATTR0]]
-; CHECK-NEXT: store i128 [[A2]], ptr [[TMP2]], align 4
-; CHECK-NEXT: store i128 [[P2]], ptr [[TMP1]], align 4
+; CHECK-NEXT: store i128 [[A2]], ptr [[TMP2]], align 16
+; CHECK-NEXT: store i128 [[P2]], ptr [[TMP1]], align 16
; CHECK-NEXT: call void @__instrumentor_pre_numeric_ind(i32 12, i32 16, i32 24, ptr [[TMP2]], ptr [[TMP1]], i64 0, i32 28) #[[ATTR0]]
; CHECK-NEXT: [[A3:%.*]] = srem i128 [[A2]], [[P2]]
-; CHECK-NEXT: store i128 [[A3]], ptr [[TMP0]], align 4
+; CHECK-NEXT: store i128 [[A3]], ptr [[TMP0]], align 16
; CHECK-NEXT: call void @__instrumentor_post_numeric_ind(i32 12, i32 16, i32 24, ptr [[TMP2]], ptr [[TMP1]], ptr [[TMP0]], i64 0, i32 -28) #[[ATTR0]]
-; CHECK-NEXT: store i128 [[A3]], ptr [[TMP2]], align 4
-; CHECK-NEXT: store i128 [[A1]], ptr [[TMP0]], align 4
+; CHECK-NEXT: store i128 [[A3]], ptr [[TMP2]], align 16
+; CHECK-NEXT: store i128 [[A1]], ptr [[TMP0]], align 16
; CHECK-NEXT: call void @__instrumentor_pre_numeric_ind(i32 12, i32 16, i32 16, ptr [[TMP2]], ptr [[TMP0]], i64 1, i32 29) #[[ATTR0]]
; CHECK-NEXT: [[A4:%.*]] = sub nsw i128 [[A3]], [[A1]]
-; CHECK-NEXT: store i128 [[A4]], ptr [[TMP1]], align 4
+; CHECK-NEXT: store i128 [[A4]], ptr [[TMP1]], align 16
; CHECK-NEXT: call void @__instrumentor_post_numeric_ind(i32 12, i32 16, i32 16, ptr [[TMP2]], ptr [[TMP0]], ptr [[TMP1]], i64 1, i32 -29) #[[ATTR0]]
-; CHECK-NEXT: store i128 [[A4]], ptr [[TMP2]], align 4
-; CHECK-NEXT: store i128 [[A2]], ptr [[TMP1]], align 4
+; CHECK-NEXT: store i128 [[A4]], ptr [[TMP2]], align 16
+; CHECK-NEXT: store i128 [[A2]], ptr [[TMP1]], align 16
; CHECK-NEXT: call void @__instrumentor_pre_numeric_ind(i32 12, i32 16, i32 30, ptr [[TMP2]], ptr [[TMP1]], i64 32, i32 30) #[[ATTR0]]
; CHECK-NEXT: [[A5:%.*]] = or disjoint i128 [[A4]], [[A2]]
-; CHECK-NEXT: store i128 [[A5]], ptr [[TMP0]], align 4
+; CHECK-NEXT: store i128 [[A5]], ptr [[TMP0]], align 16
; CHECK-NEXT: call void @__instrumentor_post_numeric_ind(i32 12, i32 16, i32 30, ptr [[TMP2]], ptr [[TMP1]], ptr [[TMP0]], i64 32, i32 -30) #[[ATTR0]]
-; CHECK-NEXT: store i128 [[A5]], ptr [[TMP2]], align 4
-; CHECK-NEXT: store i128 [[A3]], ptr [[TMP0]], align 4
+; CHECK-NEXT: store i128 [[A5]], ptr [[TMP2]], align 16
+; CHECK-NEXT: store i128 [[A3]], ptr [[TMP0]], align 16
; CHECK-NEXT: call void @__instrumentor_pre_numeric_ind(i32 12, i32 16, i32 26, ptr [[TMP2]], ptr [[TMP0]], i64 0, i32 31) #[[ATTR0]]
; CHECK-NEXT: [[A6:%.*]] = shl i128 [[A5]], [[A3]]
-; CHECK-NEXT: store i128 [[A6]], ptr [[TMP1]], align 4
+; CHECK-NEXT: store i128 [[A6]], ptr [[TMP1]], align 16
; CHECK-NEXT: call void @__instrumentor_post_numeric_ind(i32 12, i32 16, i32 26, ptr [[TMP2]], ptr [[TMP0]], ptr [[TMP1]], i64 0, i32 -31) #[[ATTR0]]
; CHECK-NEXT: ret i128 [[A6]]
;
diff --git a/llvm/test/Instrumentation/MemorySanitizer/PowerPC32/kernel-ppcle.ll b/llvm/test/Instrumentation/MemorySanitizer/PowerPC32/kernel-ppcle.ll
index 8ba033061defe..5722e382d3c22 100644
--- a/llvm/test/Instrumentation/MemorySanitizer/PowerPC32/kernel-ppcle.ll
+++ b/llvm/test/Instrumentation/MemorySanitizer/PowerPC32/kernel-ppcle.ll
@@ -229,12 +229,12 @@ define void @Store16(ptr %p, i128 %x) sanitize_memory {
; CHECK-NEXT: [[TMP15:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_store_n(ptr [[P]], i32 16)
; CHECK-NEXT: [[TMP16:%.*]] = extractvalue { ptr, ptr } [[TMP15]], 0
; CHECK-NEXT: [[TMP17:%.*]] = extractvalue { ptr, ptr } [[TMP15]], 1
-; CHECK-NEXT: store i128 [[TMP9]], ptr [[TMP16]], align 8
+; CHECK-NEXT: store i128 [[TMP9]], ptr [[TMP16]], align 16
; CHECK-NEXT: [[_MSCMP3:%.*]] = icmp ne i128 [[TMP9]], 0
; CHECK-NEXT: br i1 [[_MSCMP3]], label %[[BB11:.*]], label %[[BB16:.*]], !prof [[PROF1]]
; CHECK: [[BB11]]:
; CHECK-NEXT: [[TMP19:%.*]] = call i32 @__msan_chain_origin(i32 [[TMP12]])
-; CHECK-NEXT: store i32 [[TMP19]], ptr [[TMP17]], align 8
+; CHECK-NEXT: store i32 [[TMP19]], ptr [[TMP17]], align 16
; CHECK-NEXT: [[TMP22:%.*]] = getelementptr i32, ptr [[TMP17]], i32 1
; CHECK-NEXT: store i32 [[TMP19]], ptr [[TMP22]], align 4
; CHECK-NEXT: [[TMP20:%.*]] = getelementptr i32, ptr [[TMP17]], i32 2
@@ -243,7 +243,7 @@ define void @Store16(ptr %p, i128 %x) sanitize_memory {
; CHECK-NEXT: store i32 [[TMP19]], ptr [[TMP21]], align 4
; CHECK-NEXT: br label %[[BB16]]
; CHECK: [[BB16]]:
-; CHECK-NEXT: store i128 [[X]], ptr [[P]], align 8
+; CHECK-NEXT: store i128 [[X]], ptr [[P]], align 16
; CHECK-NEXT: ret void
;
entry:
@@ -436,12 +436,12 @@ define i128 @Load16(ptr %p) sanitize_memory {
; CHECK-NEXT: call void @__msan_warning(i32 [[TMP3]]) #[[ATTR2]]
; CHECK-NEXT: br label %[[BB5]]
; CHECK: [[BB5]]:
-; CHECK-NEXT: [[TMP9:%.*]] = load i128, ptr [[P]], align 8
+; CHECK-NEXT: [[TMP9:%.*]] = load i128, ptr [[P]], align 16
; CHECK-NEXT: [[TMP10:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_load_n(ptr [[P]], i32 16)
; CHECK-NEXT: [[TMP11:%.*]] = extractvalue { ptr, ptr } [[TMP10]], 0
; CHECK-NEXT: [[TMP12:%.*]] = extractvalue { ptr, ptr } [[TMP10]], 1
-; CHECK-NEXT: [[_MSLD:%.*]] = load i128, ptr [[TMP11]], align 8
-; CHECK-NEXT: [[TMP13:%.*]] = load i32, ptr [[TMP12]], align 8
+; CHECK-NEXT: [[_MSLD:%.*]] = load i128, ptr [[TMP11]], align 16
+; CHECK-NEXT: [[TMP13:%.*]] = load i32, ptr [[TMP12]], align 16
; CHECK-NEXT: store i128 [[_MSLD]], ptr [[RETVAL_SHADOW]], align 8
; CHECK-NEXT: store i32 [[TMP13]], ptr [[RETVAL_ORIGIN]], align 4
; CHECK-NEXT: ret i128 [[TMP9]]
diff --git a/llvm/test/Instrumentation/MemorySanitizer/byval.ll b/llvm/test/Instrumentation/MemorySanitizer/byval.ll
index 9f6a7cb189547..1ddb3a6ef6844 100644
--- a/llvm/test/Instrumentation/MemorySanitizer/byval.ll
+++ b/llvm/test/Instrumentation/MemorySanitizer/byval.ll
@@ -19,14 +19,14 @@ define i128 @ByValArgument(i32, ptr byval(i128) %p) sanitize_memory {
; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr align 8 [[TMP3]], ptr align 8 getelementptr (i8, ptr @__msan_param_tls, i64 8), i64 16, i1 false)
; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr align 4 [[TMP5]], ptr align 4 getelementptr (i8, ptr @__msan_param_origin_tls, i64 8), i64 16, i1 false)
; CHECK-NEXT: call void @llvm.donothing()
-; CHECK-NEXT: [[X:%.*]] = load i128, ptr [[P]], align 8
+; CHECK-NEXT: [[X:%.*]] = load i128, ptr [[P]], align 16
; CHECK-NEXT: [[TMP6:%.*]] = ptrtoint ptr [[P]] to i64
; CHECK-NEXT: [[TMP7:%.*]] = xor i64 [[TMP6]], 87960930222080
; CHECK-NEXT: [[TMP8:%.*]] = inttoptr i64 [[TMP7]] to ptr
; CHECK-NEXT: [[TMP9:%.*]] = add i64 [[TMP7]], 17592186044416
; CHECK-NEXT: [[TMP10:%.*]] = inttoptr i64 [[TMP9]] to ptr
-; CHECK-NEXT: [[_MSLD:%.*]] = load i128, ptr [[TMP8]], align 8
-; CHECK-NEXT: [[TMP11:%.*]] = load i32, ptr [[TMP10]], align 8
+; CHECK-NEXT: [[_MSLD:%.*]] = load i128, ptr [[TMP8]], align 16
+; CHECK-NEXT: [[TMP11:%.*]] = load i32, ptr [[TMP10]], align 16
; CHECK-NEXT: store i128 [[_MSLD]], ptr @__msan_retval_tls, align 8
; CHECK-NEXT: store i32 [[TMP11]], ptr @__msan_retval_origin_tls, align 4
; CHECK-NEXT: ret i128 [[X]]
@@ -45,9 +45,9 @@ define i128 @ByValArgumentNoSanitize(i32, ptr byval(i128) %p) {
; CHECK-NEXT: [[TMP3:%.*]] = inttoptr i64 [[TMP2]] to ptr
; CHECK-NEXT: [[TMP4:%.*]] = add i64 [[TMP2]], 17592186044416
; CHECK-NEXT: [[TMP5:%.*]] = inttoptr i64 [[TMP4]] to ptr
-; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr align 8 [[TMP3]], i8 0, i64 16, i1 false)
+; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr align 16 [[TMP3]], i8 0, i64 16, i1 false)
; CHECK-NEXT: call void @llvm.donothing()
-; CHECK-NEXT: [[X:%.*]] = load i128, ptr [[P]], align 8
+; CHECK-NEXT: [[X:%.*]] = load i128, ptr [[P]], align 16
; CHECK-NEXT: store i128 0, ptr @__msan_retval_tls, align 8
; CHECK-NEXT: store i32 0, ptr @__msan_retval_origin_tls, align 4
; CHECK-NEXT: ret i128 [[X]]
@@ -87,7 +87,7 @@ define void @ByValForwardNoSanitize(i32, ptr byval(i128) %p) {
; CHECK-NEXT: [[TMP3:%.*]] = inttoptr i64 [[TMP2]] to ptr
; CHECK-NEXT: [[TMP4:%.*]] = add i64 [[TMP2]], 17592186044416
; CHECK-NEXT: [[TMP5:%.*]] = inttoptr i64 [[TMP4]] to ptr
-; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr align 8 [[TMP3]], i8 0, i64 16, i1 false)
+; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr align 16 [[TMP3]], i8 0, i64 16, i1 false)
; CHECK-NEXT: call void @llvm.donothing()
; CHECK-NEXT: store i64 0, ptr @__msan_param_tls, align 8
; CHECK-NEXT: call void @Fn(ptr [[P]])
@@ -135,7 +135,7 @@ define void @ByValForwardByValNoSanitize(i32, ptr byval(i128) %p) {
; CHECK-NEXT: [[TMP3:%.*]] = inttoptr i64 [[TMP2]] to ptr
; CHECK-NEXT: [[TMP4:%.*]] = add i64 [[TMP2]], 17592186044416
; CHECK-NEXT: [[TMP5:%.*]] = inttoptr i64 [[TMP4]] to ptr
-; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr align 8 [[TMP3]], i8 0, i64 16, i1 false)
+; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr align 16 [[TMP3]], i8 0, i64 16, i1 false)
; CHECK-NEXT: call void @llvm.donothing()
; CHECK-NEXT: [[TMP6:%.*]] = ptrtoint ptr [[P]] to i64
; CHECK-NEXT: [[TMP7:%.*]] = xor i64 [[TMP6]], 87960930222080
diff --git a/llvm/test/Instrumentation/MemorySanitizer/msan_kernel_basic.ll b/llvm/test/Instrumentation/MemorySanitizer/msan_kernel_basic.ll
index 5d63367919d1a..e0526d56d9c2d 100644
--- a/llvm/test/Instrumentation/MemorySanitizer/msan_kernel_basic.ll
+++ b/llvm/test/Instrumentation/MemorySanitizer/msan_kernel_basic.ll
@@ -243,7 +243,7 @@ define void @Store16(ptr nocapture %p, i128 %x) nounwind uwtable sanitize_memory
; CHECK-NEXT: [[TMP13:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_store_n(ptr [[P]], i64 16)
; CHECK-NEXT: [[TMP14:%.*]] = extractvalue { ptr, ptr } [[TMP13]], 0
; CHECK-NEXT: [[TMP15:%.*]] = extractvalue { ptr, ptr } [[TMP13]], 1
-; CHECK-NEXT: store i128 [[TMP7]], ptr [[TMP14]], align 8
+; CHECK-NEXT: store i128 [[TMP7]], ptr [[TMP14]], align 16
; CHECK-NEXT: [[_MSCMP3:%.*]] = icmp ne i128 [[TMP7]], 0
; CHECK-NEXT: br i1 [[_MSCMP3]], label %[[BB10:.*]], label %[[BB16:.*]], !prof [[PROF1]]
; CHECK: [[BB10]]:
@@ -251,12 +251,12 @@ define void @Store16(ptr nocapture %p, i128 %x) nounwind uwtable sanitize_memory
; CHECK-NEXT: [[TMP18:%.*]] = zext i32 [[TMP17]] to i64
; CHECK-NEXT: [[TMP19:%.*]] = shl i64 [[TMP18]], 32
; CHECK-NEXT: [[TMP20:%.*]] = or i64 [[TMP18]], [[TMP19]]
-; CHECK-NEXT: store i64 [[TMP20]], ptr [[TMP15]], align 8
+; CHECK-NEXT: store i64 [[TMP20]], ptr [[TMP15]], align 16
; CHECK-NEXT: [[TMP21:%.*]] = getelementptr i64, ptr [[TMP15]], i32 1
; CHECK-NEXT: store i64 [[TMP20]], ptr [[TMP21]], align 8
; CHECK-NEXT: br label %[[BB16]]
; CHECK: [[BB16]]:
-; CHECK-NEXT: store i128 [[X]], ptr [[P]], align 8
+; CHECK-NEXT: store i128 [[X]], ptr [[P]], align 16
; CHECK-NEXT: ret void
;
entry:
@@ -441,12 +441,12 @@ define i128 @Load16(ptr nocapture %p) nounwind uwtable sanitize_memory {
; CHECK-NEXT: call void @__msan_warning(i32 [[TMP4]]) #[[ATTR8]]
; CHECK-NEXT: br label %[[BB4]]
; CHECK: [[BB4]]:
-; CHECK-NEXT: [[TMP7:%.*]] = load i128, ptr [[P]], align 8
+; CHECK-NEXT: [[TMP7:%.*]] = load i128, ptr [[P]], align 16
; CHECK-NEXT: [[TMP8:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_load_n(ptr [[P]], i64 16)
; CHECK-NEXT: [[TMP9:%.*]] = extractvalue { ptr, ptr } [[TMP8]], 0
; CHECK-NEXT: [[TMP10:%.*]] = extractvalue { ptr, ptr } [[TMP8]], 1
-; CHECK-NEXT: [[_MSLD:%.*]] = load i128, ptr [[TMP9]], align 8
-; CHECK-NEXT: [[TMP11:%.*]] = load i32, ptr [[TMP10]], align 8
+; CHECK-NEXT: [[_MSLD:%.*]] = load i128, ptr [[TMP9]], align 16
+; CHECK-NEXT: [[TMP11:%.*]] = load i32, ptr [[TMP10]], align 16
; CHECK-NEXT: store i128 [[_MSLD]], ptr [[RETVAL_SHADOW]], align 8
; CHECK-NEXT: store i32 [[TMP11]], ptr [[RETVAL_ORIGIN]], align 4
; CHECK-NEXT: ret i128 [[TMP7]]
@@ -490,7 +490,7 @@ define dso_local i32 @VarArgFn(ptr %fmt, ...) local_unnamed_addr sanitize_memory
; CHECK-NEXT: [[TMP9:%.*]] = alloca i8, i64 [[TMP6]], align 8
; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr align 8 [[TMP9]], ptr align 8 [[VA_ARG_ORIGIN]], i64 [[TMP8]], i1 false)
; CHECK-NEXT: call void @llvm.donothing()
-; CHECK-NEXT: [[ARGS:%.*]] = alloca [1 x %struct.__va_list_tag], align 16
+; CHECK-NEXT: [[ARGS:%.*]] = alloca [1 x [[STRUCT___VA_LIST_TAG:%.*]]], align 16
; CHECK-NEXT: call void @__msan_poison_alloca(ptr [[ARGS]], i64 24, ptr @[[GLOB0:[0-9]+]])
; CHECK-NEXT: [[TMP10:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_store_1(ptr [[ARGS]])
; CHECK-NEXT: [[TMP11:%.*]] = extractvalue { ptr, ptr } [[TMP10]], 0
diff --git a/llvm/test/Transforms/AtomicExpand/SPARC/libcalls.ll b/llvm/test/Transforms/AtomicExpand/SPARC/libcalls.ll
index 3a00ae2217c4f..53ea22b9ddf46 100644
--- a/llvm/test/Transforms/AtomicExpand/SPARC/libcalls.ll
+++ b/llvm/test/Transforms/AtomicExpand/SPARC/libcalls.ll
@@ -67,10 +67,10 @@ define i16 @test_add_i16(ptr %arg, i16 %val) {
;; 32-bit i386.
; CHECK-LABEL: @test_load_i128(
-; CHECK: %1 = alloca i128, align 8
+; CHECK: %1 = alloca i128, align 16
; CHECK: call void @llvm.lifetime.start.p0(ptr %1)
; CHECK: call void @__atomic_load(i32 16, ptr %arg, ptr %1, i32 5)
-; CHECK: %2 = load i128, ptr %1, align 8
+; CHECK: %2 = load i128, ptr %1, align 16
; CHECK: call void @llvm.lifetime.end.p0(ptr %1)
; CHECK: ret i128 %2
define i128 @test_load_i128(ptr %arg) {
@@ -79,9 +79,9 @@ define i128 @test_load_i128(ptr %arg) {
}
; CHECK-LABEL: @test_store_i128(
-; CHECK: %1 = alloca i128, align 8
+; CHECK: %1 = alloca i128, align 16
; CHECK: call void @llvm.lifetime.start.p0(ptr %1)
-; CHECK: store i128 %val, ptr %1, align 8
+; CHECK: store i128 %val, ptr %1, align 16
; CHECK: call void @__atomic_store(i32 16, ptr %arg, ptr %1, i32 5)
; CHECK: call void @llvm.lifetime.end.p0(ptr %1)
; CHECK: ret void
@@ -91,14 +91,14 @@ define void @test_store_i128(ptr %arg, i128 %val) {
}
; CHECK-LABEL: @test_exchange_i128(
-; CHECK: %1 = alloca i128, align 8
+; CHECK: %1 = alloca i128, align 16
; CHECK: call void @llvm.lifetime.start.p0(ptr %1)
-; CHECK: store i128 %val, ptr %1, align 8
-; CHECK: %2 = alloca i128, align 8
+; CHECK: store i128 %val, ptr %1, align 16
+; CHECK: %2 = alloca i128, align 16
; CHECK: call void @llvm.lifetime.start.p0(ptr %2)
; CHECK: call void @__atomic_exchange(i32 16, ptr %arg, ptr %1, ptr %2, i32 5)
; CHECK: call void @llvm.lifetime.end.p0(ptr %1)
-; CHECK: %3 = load i128, ptr %2, align 8
+; CHECK: %3 = load i128, ptr %2, align 16
; CHECK: call void @llvm.lifetime.end.p0(ptr %2)
; CHECK: ret i128 %3
define i128 @test_exchange_i128(ptr %arg, i128 %val) {
@@ -107,15 +107,15 @@ define i128 @test_exchange_i128(ptr %arg, i128 %val) {
}
; CHECK-LABEL: @test_cmpxchg_i128(
-; CHECK: %1 = alloca i128, align 8
+; CHECK: %1 = alloca i128, align 16
; CHECK: call void @llvm.lifetime.start.p0(ptr %1)
-; CHECK: store i128 %old, ptr %1, align 8
-; CHECK: %2 = alloca i128, align 8
+; CHECK: store i128 %old, ptr %1, align 16
+; CHECK: %2 = alloca i128, align 16
; CHECK: call void @llvm.lifetime.start.p0(ptr %2)
-; CHECK: store i128 %new, ptr %2, align 8
+; CHECK: store i128 %new, ptr %2, align 16
; CHECK: %3 = call zeroext i1 @__atomic_compare_exchange(i32 16, ptr %arg, ptr %1, ptr %2, i32 5, i32 0)
; CHECK: call void @llvm.lifetime.end.p0(ptr %2)
-; CHECK: %4 = load i128, ptr %1, align 8
+; CHECK: %4 = load i128, ptr %1, align 16
; CHECK: call void @llvm.lifetime.end.p0(ptr %1)
; CHECK: %5 = insertvalue { i128, i1 } poison, i128 %4, 0
; CHECK: %6 = insertvalue { i128, i1 } %5, i1 %3, 1
@@ -132,24 +132,24 @@ define i128 @test_cmpxchg_i128(ptr %arg, i128 %old, i128 %new) {
; loop, which then itself expands into a libcall.
; CHECK-LABEL: @test_add_i128(
-; CHECK: %1 = alloca i128, align 8
-; CHECK: %2 = alloca i128, align 8
-; CHECK: %3 = alloca i128, align 8
+; CHECK: %1 = alloca i128, align 16
+; CHECK: %2 = alloca i128, align 16
+; CHECK: %3 = alloca i128, align 16
; CHECK: call void @llvm.lifetime.start.p0(ptr %3)
; CHECK: call void @__atomic_load(i32 16, ptr %arg, ptr %3, i32 0)
-; CHECK: %4 = load i128, ptr %3, align 8
+; CHECK: %4 = load i128, ptr %3, align 16
; CHECK: call void @llvm.lifetime.end.p0(ptr %3)
; CHECK: br label %atomicrmw.start
; CHECK:atomicrmw.start:
; CHECK: %loaded = phi i128 [ %4, %0 ], [ %newloaded, %atomicrmw.start ]
; CHECK: %new = add i128 %loaded, %val
; CHECK: call void @llvm.lifetime.start.p0(ptr %1)
-; CHECK: store i128 %loaded, ptr %1, align 8
+; CHECK: store i128 %loaded, ptr %1, align 16
; CHECK: call void @llvm.lifetime.start.p0(ptr %2)
-; CHECK: store i128 %new, ptr %2, align 8
+; CHECK: store i128 %new, ptr %2, align 16
; CHECK: %5 = call zeroext i1 @__atomic_compare_exchange(i32 16, ptr %arg, ptr %1, ptr %2, i32 5, i32 5)
; CHECK: call void @llvm.lifetime.end.p0(ptr %2)
-; CHECK: %6 = load i128, ptr %1, align 8
+; CHECK: %6 = load i128, ptr %1, align 16
; CHECK: call void @llvm.lifetime.end.p0(ptr %1)
; CHECK: %7 = insertvalue { i128, i1 } poison, i128 %6, 0
; CHECK: %8 = insertvalue { i128, i1 } %7, i1 %5, 1
@@ -205,9 +205,9 @@ define ptr @test_cmpxchg_ptr(ptr %arg, ptr %old, ptr %new) {
;; ...and for a non-integer type of large size too.
; CHECK-LABEL: @test_store_fp128
-; CHECK: %1 = alloca fp128, align 8
+; CHECK: %1 = alloca fp128, align 16
; CHECK: call void @llvm.lifetime.start.p0(ptr %1)
-; CHECK: store fp128 %val, ptr %1, align 8
+; CHECK: store fp128 %val, ptr %1, align 16
; CHECK: call void @__atomic_store(i32 16, ptr %arg, ptr %1, i32 5)
; CHECK: call void @llvm.lifetime.end.p0(ptr %1)
; CHECK: ret void
diff --git a/llvm/test/Transforms/HotColdSplit/lifetime-markers-on-inputs-2.ll b/llvm/test/Transforms/HotColdSplit/lifetime-markers-on-inputs-2.ll
index da7a9b8d7531b..c5da2ff2eb47a 100644
--- a/llvm/test/Transforms/HotColdSplit/lifetime-markers-on-inputs-2.ll
+++ b/llvm/test/Transforms/HotColdSplit/lifetime-markers-on-inputs-2.ll
@@ -37,7 +37,7 @@ declare void @use(ptr)
define void @only_lifetime_start_is_cold(i1 %arg) {
; CHECK-LABEL: @only_lifetime_start_is_cold(
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[LOCAL1:%.*]] = alloca i256, align 8
+; CHECK-NEXT: [[LOCAL1:%.*]] = alloca i256, align 16
; CHECK-NEXT: br i1 [[ARG:%.*]], label [[CODEREPL:%.*]], label [[NO_EXTRACT1:%.*]]
; CHECK: codeRepl:
; CHECK-NEXT: call void @llvm.lifetime.start.p0(ptr [[LOCAL1]])
@@ -95,7 +95,7 @@ exit:
define void @only_lifetime_end_is_cold(i1 %arg) {
; CHECK-LABEL: @only_lifetime_end_is_cold(
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[LOCAL1:%.*]] = alloca i256, align 8
+; CHECK-NEXT: [[LOCAL1:%.*]] = alloca i256, align 16
; CHECK-NEXT: call void @llvm.lifetime.start.p0(ptr [[LOCAL1]])
; CHECK-NEXT: br i1 [[ARG:%.*]], label [[NO_EXTRACT1:%.*]], label [[CODEREPL:%.*]]
; CHECK: no-extract1:
@@ -133,7 +133,7 @@ exit:
define void @do_not_lift_lifetime_end(i1 %arg) {
; CHECK-LABEL: @do_not_lift_lifetime_end(
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[LOCAL1:%.*]] = alloca i256, align 8
+; CHECK-NEXT: [[LOCAL1:%.*]] = alloca i256, align 16
; CHECK-NEXT: call void @llvm.lifetime.start.p0(ptr [[LOCAL1]])
; CHECK-NEXT: br label [[HEADER:%.*]]
; CHECK: header:
diff --git a/llvm/test/Transforms/InferAlignment/irregular-size.ll b/llvm/test/Transforms/InferAlignment/irregular-size.ll
index 9413c8ac5be46..93ff9380ce646 100644
--- a/llvm/test/Transforms/InferAlignment/irregular-size.ll
+++ b/llvm/test/Transforms/InferAlignment/irregular-size.ll
@@ -4,9 +4,9 @@
define void @non_pow2_size(i177 %X) {
; CHECK-LABEL: define void @non_pow2_size
; CHECK-SAME: (i177 [[X:%.*]]) {
-; CHECK-NEXT: [[A:%.*]] = alloca i177, align 8
-; CHECK-NEXT: [[L1:%.*]] = load i177, ptr [[A]], align 8
-; CHECK-NEXT: store i177 [[X]], ptr [[A]], align 8
+; CHECK-NEXT: [[A:%.*]] = alloca i177, align 16
+; CHECK-NEXT: [[L1:%.*]] = load i177, ptr [[A]], align 16
+; CHECK-NEXT: store i177 [[X]], ptr [[A]], align 16
; CHECK-NEXT: ret void
;
%A = alloca i177, align 1
diff --git a/llvm/test/Transforms/InstCombine/apint-and.ll b/llvm/test/Transforms/InstCombine/apint-and.ll
index c5f2087ee02fd..5ef0925c3409a 100644
--- a/llvm/test/Transforms/InstCombine/apint-and.ll
+++ b/llvm/test/Transforms/InstCombine/apint-and.ll
@@ -105,7 +105,7 @@ define i117 @test12(i117 %A, ptr %P) {
; CHECK-LABEL: @test12(
; CHECK-NEXT: [[TMP1:%.*]] = and i117 [[A:%.*]], -4
; CHECK-NEXT: [[C:%.*]] = xor i117 [[TMP1]], 15
-; CHECK-NEXT: store i117 [[C]], ptr [[P:%.*]], align 4
+; CHECK-NEXT: store i117 [[C]], ptr [[P:%.*]], align 16
; CHECK-NEXT: ret i117 3
;
%B = or i117 %A, 3
diff --git a/llvm/test/Transforms/InstCombine/load-store-forward.ll b/llvm/test/Transforms/InstCombine/load-store-forward.ll
index 6a0897ff75036..2e18334416455 100644
--- a/llvm/test/Transforms/InstCombine/load-store-forward.ll
+++ b/llvm/test/Transforms/InstCombine/load-store-forward.ll
@@ -458,7 +458,7 @@ define i32 @load_after_memset_0_clobber(ptr %a) {
define i256 @load_after_memset_0_too_small(ptr %a) {
; CHECK-LABEL: @load_after_memset_0_too_small(
; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr noundef nonnull align 1 dereferenceable(16) [[A:%.*]], i8 0, i64 16, i1 false)
-; CHECK-NEXT: [[V:%.*]] = load i256, ptr [[A]], align 4
+; CHECK-NEXT: [[V:%.*]] = load i256, ptr [[A]], align 16
; CHECK-NEXT: ret i256 [[V]]
;
call void @llvm.memset.p0.i64(ptr %a, i8 0, i64 16, i1 false)
@@ -469,7 +469,7 @@ define i256 @load_after_memset_0_too_small(ptr %a) {
define i129 @load_after_memset_0_too_small_by_one_bit(ptr %a) {
; CHECK-LABEL: @load_after_memset_0_too_small_by_one_bit(
; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr noundef nonnull align 1 dereferenceable(16) [[A:%.*]], i8 0, i64 16, i1 false)
-; CHECK-NEXT: [[V:%.*]] = load i129, ptr [[A]], align 4
+; CHECK-NEXT: [[V:%.*]] = load i129, ptr [[A]], align 16
; CHECK-NEXT: ret i129 [[V]]
;
call void @llvm.memset.p0.i64(ptr %a, i8 0, i64 16, i1 false)
diff --git a/llvm/test/Transforms/InstCombine/or-xor.ll b/llvm/test/Transforms/InstCombine/or-xor.ll
index b05ff15b8b3c8..8920711e2ca09 100644
--- a/llvm/test/Transforms/InstCombine/or-xor.ll
+++ b/llvm/test/Transforms/InstCombine/or-xor.ll
@@ -125,7 +125,7 @@ define i8 @test5_extra_use_not(i8 %x, i8 %y, ptr %dst) {
define i65 @test5_extra_use_xor(i65 %x, i65 %y, ptr %dst) {
; CHECK-LABEL: @test5_extra_use_xor(
; CHECK-NEXT: [[XOR:%.*]] = xor i65 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT: store i65 [[XOR]], ptr [[DST:%.*]], align 4
+; CHECK-NEXT: store i65 [[XOR]], ptr [[DST:%.*]], align 16
; CHECK-NEXT: [[TMP1:%.*]] = and i65 [[X]], [[Y]]
; CHECK-NEXT: [[Z:%.*]] = xor i65 [[TMP1]], -1
; CHECK-NEXT: ret i65 [[Z]]
diff --git a/llvm/test/Transforms/InstCombine/select.ll b/llvm/test/Transforms/InstCombine/select.ll
index a1e84fad9a827..aaedf9d0e43fa 100644
--- a/llvm/test/Transforms/InstCombine/select.ll
+++ b/llvm/test/Transforms/InstCombine/select.ll
@@ -1348,11 +1348,11 @@ define ptr @test85(i1 %flag) {
; CHECK-LABEL: define ptr @test85(
; CHECK-SAME: i1 [[FLAG:%.*]]) {
; CHECK-NEXT: [[X:%.*]] = alloca [2 x ptr], align 8
-; CHECK-NEXT: [[Y:%.*]] = alloca i128, align 8
+; CHECK-NEXT: [[Y:%.*]] = alloca i128, align 16
; CHECK-NEXT: call void @scribble_on_i128(ptr nonnull [[X]])
; CHECK-NEXT: call void @scribble_on_i128(ptr nonnull [[Y]])
-; CHECK-NEXT: [[T:%.*]] = load i128, ptr [[X]], align 4
-; CHECK-NEXT: store i128 [[T]], ptr [[Y]], align 4
+; CHECK-NEXT: [[T:%.*]] = load i128, ptr [[X]], align 16
+; CHECK-NEXT: store i128 [[T]], ptr [[Y]], align 16
; CHECK-NEXT: [[X_VAL:%.*]] = load ptr, ptr [[X]], align 8
; CHECK-NEXT: [[Y_VAL:%.*]] = load ptr, ptr [[Y]], align 8
; CHECK-NEXT: [[V:%.*]] = select i1 [[FLAG]], ptr [[X_VAL]], ptr [[Y_VAL]]
@@ -1376,14 +1376,13 @@ define i128 @test86(i1 %flag) {
; CHECK-LABEL: define i128 @test86(
; CHECK-SAME: i1 [[FLAG:%.*]]) {
; CHECK-NEXT: [[X:%.*]] = alloca [2 x ptr], align 8
-; CHECK-NEXT: [[Y:%.*]] = alloca i128, align 8
+; CHECK-NEXT: [[Y:%.*]] = alloca i128, align 16
; CHECK-NEXT: call void @scribble_on_i128(ptr nonnull [[X]])
; CHECK-NEXT: call void @scribble_on_i128(ptr nonnull [[Y]])
; CHECK-NEXT: [[T:%.*]] = load ptr, ptr [[X]], align 8
; CHECK-NEXT: store ptr [[T]], ptr [[Y]], align 8
-; CHECK-NEXT: [[X_VAL:%.*]] = load i128, ptr [[X]], align 4
-; CHECK-NEXT: [[Y_VAL:%.*]] = load i128, ptr [[Y]], align 4
-; CHECK-NEXT: [[V:%.*]] = select i1 [[FLAG]], i128 [[X_VAL]], i128 [[Y_VAL]]
+; CHECK-NEXT: [[P:%.*]] = select i1 [[FLAG]], ptr [[X]], ptr [[Y]]
+; CHECK-NEXT: [[V:%.*]] = load i128, ptr [[P]], align 16
; CHECK-NEXT: ret i128 [[V]]
;
%x = alloca [2 x ptr]
diff --git a/llvm/test/Transforms/InstCombine/shift.ll b/llvm/test/Transforms/InstCombine/shift.ll
index d63ba48830068..701ee07b36d84 100644
--- a/llvm/test/Transforms/InstCombine/shift.ll
+++ b/llvm/test/Transforms/InstCombine/shift.ll
@@ -1740,19 +1740,19 @@ define i177 @lshr_out_of_range2(i177 %Y, ptr %A2, ptr %ptr) {
; https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=5032
define void @ashr_out_of_range(ptr %A) {
; CHECK-LABEL: @ashr_out_of_range(
-; CHECK-NEXT: [[L:%.*]] = load i177, ptr [[A:%.*]], align 4
+; CHECK-NEXT: [[L:%.*]] = load i177, ptr [[A:%.*]], align 16
; CHECK-NEXT: [[TMP1:%.*]] = icmp eq i177 [[L]], -1
; CHECK-NEXT: [[TMP2:%.*]] = select i1 [[TMP1]], i64 -1, i64 -2
-; CHECK-NEXT: [[G11:%.*]] = getelementptr [24 x i8], ptr [[A]], i64 [[TMP2]]
-; CHECK-NEXT: [[L7:%.*]] = load i177, ptr [[G11]], align 4
+; CHECK-NEXT: [[G11:%.*]] = getelementptr [32 x i8], ptr [[A]], i64 [[TMP2]]
+; CHECK-NEXT: [[L7:%.*]] = load i177, ptr [[G11]], align 16
; CHECK-NEXT: [[L7_FROZEN:%.*]] = freeze i177 [[L7]]
; CHECK-NEXT: [[C171:%.*]] = icmp slt i177 [[L7_FROZEN]], 0
; CHECK-NEXT: [[C17:%.*]] = and i1 [[TMP1]], [[C171]]
; CHECK-NEXT: [[TMP3:%.*]] = sext i1 [[C17]] to i64
-; CHECK-NEXT: [[G62:%.*]] = getelementptr [24 x i8], ptr [[G11]], i64 [[TMP3]]
+; CHECK-NEXT: [[G62:%.*]] = getelementptr [32 x i8], ptr [[G11]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP4:%.*]] = icmp eq i177 [[L7_FROZEN]], -1
; CHECK-NEXT: [[B28:%.*]] = select i1 [[TMP4]], i177 0, i177 [[L7_FROZEN]]
-; CHECK-NEXT: store i177 [[B28]], ptr [[G62]], align 4
+; CHECK-NEXT: store i177 [[B28]], ptr [[G62]], align 16
; CHECK-NEXT: ret void
;
%L = load i177, ptr %A
@@ -1780,10 +1780,10 @@ define void @ashr_out_of_range_1(ptr %A) {
; CHECK-NEXT: [[TMP1:%.*]] = icmp eq i177 [[L_FROZEN]], -1
; CHECK-NEXT: [[TMP2:%.*]] = trunc i177 [[L_FROZEN]] to i64
; CHECK-NEXT: [[TMP3:%.*]] = select i1 [[TMP1]], i64 0, i64 [[TMP2]]
-; CHECK-NEXT: [[TMP4:%.*]] = getelementptr [24 x i8], ptr [[A]], i64 [[TMP3]]
-; CHECK-NEXT: [[G11:%.*]] = getelementptr i8, ptr [[TMP4]], i64 -24
+; CHECK-NEXT: [[TMP4:%.*]] = getelementptr [32 x i8], ptr [[A]], i64 [[TMP3]]
+; CHECK-NEXT: [[G11:%.*]] = getelementptr i8, ptr [[TMP4]], i64 -32
; CHECK-NEXT: [[TMP5:%.*]] = sext i1 [[TMP1]] to i64
-; CHECK-NEXT: [[G62:%.*]] = getelementptr [24 x i8], ptr [[G11]], i64 [[TMP5]]
+; CHECK-NEXT: [[G62:%.*]] = getelementptr [32 x i8], ptr [[G11]], i64 [[TMP5]]
; CHECK-NEXT: [[TMP6:%.*]] = icmp eq i177 [[L_FROZEN]], -1
; CHECK-NEXT: [[B28:%.*]] = select i1 [[TMP6]], i177 0, i177 [[L_FROZEN]]
; CHECK-NEXT: store i177 [[B28]], ptr [[G62]], align 4
diff --git a/llvm/test/Transforms/InstCombine/xor.ll b/llvm/test/Transforms/InstCombine/xor.ll
index 3abaf74285cc0..b2b1c4787f657 100644
--- a/llvm/test/Transforms/InstCombine/xor.ll
+++ b/llvm/test/Transforms/InstCombine/xor.ll
@@ -1304,7 +1304,7 @@ define i67 @xor_orn_commute3_1use(i67 %pa, i67 %pb, ptr %s) {
; CHECK-NEXT: [[B:%.*]] = udiv i67 42, [[PB:%.*]]
; CHECK-NEXT: [[NOTA:%.*]] = xor i67 [[A]], -1
; CHECK-NEXT: [[L:%.*]] = or i67 [[B]], [[NOTA]]
-; CHECK-NEXT: store i67 [[L]], ptr [[S:%.*]], align 4
+; CHECK-NEXT: store i67 [[L]], ptr [[S:%.*]], align 16
; CHECK-NEXT: [[Z:%.*]] = xor i67 [[A]], [[L]]
; CHECK-NEXT: ret i67 [[Z]]
;
diff --git a/llvm/test/Transforms/LoopIdiom/X86/unordered-atomic-memcpy.ll b/llvm/test/Transforms/LoopIdiom/X86/unordered-atomic-memcpy.ll
index df12769b2527f..35b56ce18b95b 100644
--- a/llvm/test/Transforms/LoopIdiom/X86/unordered-atomic-memcpy.ll
+++ b/llvm/test/Transforms/LoopIdiom/X86/unordered-atomic-memcpy.ll
@@ -544,8 +544,8 @@ for.end: ; preds = %for.body, %entry
define void @test9(i64 %Size) nounwind ssp {
; CHECK-LABEL: @test9(
; CHECK-NEXT: bb.nph:
-; CHECK-NEXT: [[BASE:%.*]] = alloca i128, i32 10000, align 8
-; CHECK-NEXT: [[DEST:%.*]] = alloca i128, i32 10000, align 8
+; CHECK-NEXT: [[BASE:%.*]] = alloca i128, i32 10000, align 16
+; CHECK-NEXT: [[DEST:%.*]] = alloca i128, i32 10000, align 16
; CHECK-NEXT: [[TMP0:%.*]] = shl nuw i64 [[SIZE:%.*]], 4
; CHECK-NEXT: call void @llvm.memcpy.element.unordered.atomic.p0.p0.i64(ptr align 16 [[DEST]], ptr align 16 [[BASE]], i64 [[TMP0]], i32 16)
; CHECK-NEXT: br label [[FOR_BODY:%.*]]
@@ -583,8 +583,8 @@ for.end: ; preds = %for.body, %entry
define void @test10(i64 %Size) nounwind ssp {
; CHECK-LABEL: @test10(
; CHECK-NEXT: bb.nph:
-; CHECK-NEXT: [[BASE:%.*]] = alloca i256, i32 10000, align 8
-; CHECK-NEXT: [[DEST:%.*]] = alloca i256, i32 10000, align 8
+; CHECK-NEXT: [[BASE:%.*]] = alloca i256, i32 10000, align 16
+; CHECK-NEXT: [[DEST:%.*]] = alloca i256, i32 10000, align 16
; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body:
; CHECK-NEXT: [[INDVAR:%.*]] = phi i64 [ 0, [[BB_NPH:%.*]] ], [ [[INDVAR_NEXT:%.*]], [[FOR_BODY]] ]
diff --git a/llvm/test/Transforms/LowerMatrixIntrinsics/unary.ll b/llvm/test/Transforms/LowerMatrixIntrinsics/unary.ll
index 5d7777245ebed..697420a2e999a 100644
--- a/llvm/test/Transforms/LowerMatrixIntrinsics/unary.ll
+++ b/llvm/test/Transforms/LowerMatrixIntrinsics/unary.ll
@@ -218,7 +218,7 @@ define void @bitcast_2x2_v4f64_to_v8i32(ptr %in, ptr %out) {
define void @bitcast_2x2_i256_to_v4i64(ptr %in, ptr %out) {
; CHECK-LABEL: @bitcast_2x2_i256_to_v4i64(
-; CHECK-NEXT: [[INV:%.*]] = load i256, ptr [[IN:%.*]], align 4
+; CHECK-NEXT: [[INV:%.*]] = load i256, ptr [[IN:%.*]], align 16
; CHECK-NEXT: [[OP:%.*]] = bitcast i256 [[INV]] to <4 x double>
; CHECK-NEXT: [[SPLIT:%.*]] = shufflevector <4 x double> [[OP]], <4 x double> poison, <2 x i32> <i32 0, i32 1>
; CHECK-NEXT: [[SPLIT1:%.*]] = shufflevector <4 x double> [[OP]], <4 x double> poison, <2 x i32> <i32 2, i32 3>
@@ -240,7 +240,7 @@ define void @bitcast_2x2_4i64_to_i256(ptr %in, ptr %out) {
; CHECK-NEXT: [[COL_LOAD1:%.*]] = load <2 x double>, ptr [[VEC_GEP]], align 8
; CHECK-NEXT: [[TMP1:%.*]] = shufflevector <2 x double> [[COL_LOAD]], <2 x double> [[COL_LOAD1]], <4 x i32> <i32 0, i32 1, i32 2, i32 3>
; CHECK-NEXT: [[OP:%.*]] = bitcast <4 x double> [[TMP1]] to i256
-; CHECK-NEXT: store i256 [[OP]], ptr [[OUT:%.*]], align 4
+; CHECK-NEXT: store i256 [[OP]], ptr [[OUT:%.*]], align 16
; CHECK-NEXT: ret void
;
%inv = call <4 x double> @llvm.matrix.column.major.load(ptr %in, i64 2, i1 false, i32 2, i32 2)
diff --git a/llvm/test/Transforms/SCCP/apint-bigint2.ll b/llvm/test/Transforms/SCCP/apint-bigint2.ll
index c5e7643cdd5ab..ea47a94f6d350 100644
--- a/llvm/test/Transforms/SCCP/apint-bigint2.ll
+++ b/llvm/test/Transforms/SCCP/apint-bigint2.ll
@@ -24,7 +24,7 @@ define i101 @large_aggregate() {
; CHECK-NEXT: [[D:%.*]] = and i101 undef, 1
; CHECK-NEXT: [[DD:%.*]] = or i101 [[D]], 1
; CHECK-NEXT: [[G:%.*]] = getelementptr i101, ptr getelementptr inbounds nuw (i8, ptr @Y, i64 80), i101 [[DD]]
-; CHECK-NEXT: [[L3:%.*]] = load i101, ptr [[G]], align 4
+; CHECK-NEXT: [[L3:%.*]] = load i101, ptr [[G]], align 16
; CHECK-NEXT: ret i101 [[L3]]
;
%B = load i101, ptr undef
@@ -41,7 +41,7 @@ define i101 @large_aggregate_2() {
; CHECK-NEXT: [[D:%.*]] = and i101 undef, 1
; CHECK-NEXT: [[DD:%.*]] = or i101 [[D]], 1
; CHECK-NEXT: [[G:%.*]] = getelementptr i101, ptr getelementptr inbounds nuw (i8, ptr @Y, i64 80), i101 [[DD]]
-; CHECK-NEXT: [[L3:%.*]] = load i101, ptr [[G]], align 4
+; CHECK-NEXT: [[L3:%.*]] = load i101, ptr [[G]], align 16
; CHECK-NEXT: ret i101 [[L3]]
;
%D = and i101 undef, 1
diff --git a/llvm/test/Transforms/SCCP/ub-shift.ll b/llvm/test/Transforms/SCCP/ub-shift.ll
index ae2739d0445d0..4810c16b5685c 100644
--- a/llvm/test/Transforms/SCCP/ub-shift.ll
+++ b/llvm/test/Transforms/SCCP/ub-shift.ll
@@ -23,10 +23,10 @@ define void @shift_undef_64(ptr %p) {
define void @shift_undef_65(ptr %p) {
; CHECK-LABEL: @shift_undef_65(
-; CHECK-NEXT: store i65 0, ptr [[P:%.*]], align 4
-; CHECK-NEXT: store i65 0, ptr [[P]], align 4
+; CHECK-NEXT: store i65 0, ptr [[P:%.*]], align 16
+; CHECK-NEXT: store i65 0, ptr [[P]], align 16
; CHECK-NEXT: [[R3:%.*]] = shl nuw nsw i65 1, -18446744073709551615
-; CHECK-NEXT: store i65 [[R3]], ptr [[P]], align 4
+; CHECK-NEXT: store i65 [[R3]], ptr [[P]], align 16
; CHECK-NEXT: ret void
;
%r1 = lshr i65 2, 18446744073709551617
@@ -43,10 +43,10 @@ define void @shift_undef_65(ptr %p) {
define void @shift_undef_256(ptr %p) {
; CHECK-LABEL: @shift_undef_256(
-; CHECK-NEXT: store i256 0, ptr [[P:%.*]], align 4
-; CHECK-NEXT: store i256 0, ptr [[P]], align 4
+; CHECK-NEXT: store i256 0, ptr [[P:%.*]], align 16
+; CHECK-NEXT: store i256 0, ptr [[P]], align 16
; CHECK-NEXT: [[R3:%.*]] = shl nuw nsw i256 1, 18446744073709551619
-; CHECK-NEXT: store i256 [[R3]], ptr [[P]], align 4
+; CHECK-NEXT: store i256 [[R3]], ptr [[P]], align 16
; CHECK-NEXT: ret void
;
%r1 = lshr i256 2, 18446744073709551617
@@ -63,10 +63,10 @@ define void @shift_undef_256(ptr %p) {
define void @shift_undef_511(ptr %p) {
; CHECK-LABEL: @shift_undef_511(
-; CHECK-NEXT: store i511 0, ptr [[P:%.*]], align 4
-; CHECK-NEXT: store i511 -1, ptr [[P]], align 4
+; CHECK-NEXT: store i511 0, ptr [[P:%.*]], align 16
+; CHECK-NEXT: store i511 -1, ptr [[P]], align 16
; CHECK-NEXT: [[R3:%.*]] = shl nuw nsw i511 -3, 1208925819614629174706180
-; CHECK-NEXT: store i511 [[R3]], ptr [[P]], align 4
+; CHECK-NEXT: store i511 [[R3]], ptr [[P]], align 16
; CHECK-NEXT: ret void
;
%r1 = lshr i511 -1, 1208925819614629174706276 ; 2^80 + 100
diff --git a/llvm/test/Transforms/SimplifyCFG/jump-threading-max-jump-threading-live-blocks.ll b/llvm/test/Transforms/SimplifyCFG/jump-threading-max-jump-threading-live-blocks.ll
index 686869348019d..92bc3b54015e7 100644
--- a/llvm/test/Transforms/SimplifyCFG/jump-threading-max-jump-threading-live-blocks.ll
+++ b/llvm/test/Transforms/SimplifyCFG/jump-threading-max-jump-threading-live-blocks.ll
@@ -21,11 +21,11 @@ define void @testB(ptr %ptrA, ptr %ptrB, i64 %a, i64 %b, i64 %c) {
; CHECK_LIMIT_3-NEXT: br i1 [[COND2]], label %[[IFB_ARM1:.*]], label %[[IFB_ARM2:.*]]
; CHECK_LIMIT_3: [[IFB_ARM1]]:
; CHECK_LIMIT_3-NEXT: [[PTR_ARM1:%.*]] = getelementptr i64, ptr [[PTRB]], i64 8
-; CHECK_LIMIT_3-NEXT: store i128 0, ptr [[PTR_ARM1]], align 4
+; CHECK_LIMIT_3-NEXT: store i128 0, ptr [[PTR_ARM1]], align 16
; CHECK_LIMIT_3-NEXT: br label %[[IFB_JOIN:.*]]
; CHECK_LIMIT_3: [[IFB_ARM2]]:
; CHECK_LIMIT_3-NEXT: [[PTR_ARM2:%.*]] = getelementptr i64, ptr [[PTRB]], i64 16
-; CHECK_LIMIT_3-NEXT: store i128 0, ptr [[PTR_ARM2]], align 4
+; CHECK_LIMIT_3-NEXT: store i128 0, ptr [[PTR_ARM2]], align 16
; CHECK_LIMIT_3-NEXT: br label %[[IFB_JOIN]]
; CHECK_LIMIT_3: [[IFB_JOIN]]:
; CHECK_LIMIT_3-NEXT: [[PTRC:%.*]] = phi ptr [ [[PTR_ARM1]], %[[IFB_ARM1]] ], [ [[PTR_ARM2]], %[[IFB_ARM2]] ]
@@ -45,11 +45,11 @@ define void @testB(ptr %ptrA, ptr %ptrB, i64 %a, i64 %b, i64 %c) {
; CHECK_LIMIT_4-NEXT: br i1 [[COND2]], label %[[IFB_ARM1:.*]], label %[[IFB_ARM2:.*]]
; CHECK_LIMIT_4: [[IFB_ARM1]]:
; CHECK_LIMIT_4-NEXT: [[PTR_ARM1:%.*]] = getelementptr i64, ptr [[PTRB]], i64 8
-; CHECK_LIMIT_4-NEXT: store i128 0, ptr [[PTR_ARM1]], align 4
+; CHECK_LIMIT_4-NEXT: store i128 0, ptr [[PTR_ARM1]], align 16
; CHECK_LIMIT_4-NEXT: br label %[[IFB_JOIN:.*]]
; CHECK_LIMIT_4: [[IFB_ARM2]]:
; CHECK_LIMIT_4-NEXT: [[PTR_ARM2:%.*]] = getelementptr i64, ptr [[PTRB]], i64 16
-; CHECK_LIMIT_4-NEXT: store i128 0, ptr [[PTR_ARM2]], align 4
+; CHECK_LIMIT_4-NEXT: store i128 0, ptr [[PTR_ARM2]], align 16
; CHECK_LIMIT_4-NEXT: br label %[[IFB_JOIN]]
; CHECK_LIMIT_4: [[IFB_JOIN]]:
; CHECK_LIMIT_4-NEXT: [[PTRC:%.*]] = phi ptr [ [[PTR_ARM1]], %[[IFB_ARM1]] ], [ [[PTR_ARM2]], %[[IFB_ARM2]] ]
diff --git a/llvm/test/tools/llubi/loadstore_be.ll b/llvm/test/tools/llubi/loadstore_be.ll
index fe2733dc8149c..7dbdfbea68726 100644
--- a/llvm/test/tools/llubi/loadstore_be.ll
+++ b/llvm/test/tools/llubi/loadstore_be.ll
@@ -165,7 +165,7 @@ define void @main() {
store ptr %alloc_ptr_array, ptr %ptr_array_2
%ptr_array_2_middle = getelementptr i8, ptr %alloc_ptr_array, i64 20
store i8 0, ptr %ptr_array_2_middle
- %mixed_alloc = load b256, ptr %alloc_ptr_array
+ %mixed_alloc = load b256, ptr %alloc_ptr_array, align 8
ret void
}
diff --git a/llvm/test/tools/llubi/loadstore_le.ll b/llvm/test/tools/llubi/loadstore_le.ll
index 9c434461380cc..65ba3c97b778e 100644
--- a/llvm/test/tools/llubi/loadstore_le.ll
+++ b/llvm/test/tools/llubi/loadstore_le.ll
@@ -166,7 +166,7 @@ define void @main() {
store ptr %alloc_ptr_array, ptr %ptr_array_2
%ptr_array_2_middle = getelementptr i8, ptr %alloc_ptr_array, i64 20
store i8 0, ptr %ptr_array_2_middle
- %mixed_alloc = load b256, ptr %alloc_ptr_array
+ %mixed_alloc = load b256, ptr %alloc_ptr_array, align 8
ret void
}
diff --git a/llvm/unittests/Frontend/OpenMPIRBuilderTest.cpp b/llvm/unittests/Frontend/OpenMPIRBuilderTest.cpp
index 084dcb0a5847f..55e82b8a2aa29 100644
--- a/llvm/unittests/Frontend/OpenMPIRBuilderTest.cpp
+++ b/llvm/unittests/Frontend/OpenMPIRBuilderTest.cpp
@@ -7421,7 +7421,7 @@ TEST_F(OpenMPIRBuilderTest, CreateTask) {
ConstantInt *SharedsSize =
dyn_cast<ConstantInt>(TaskAllocCall->getOperand(4));
EXPECT_EQ(SharedsSize->getSExtValue(),
- 24); // 64-bit pointer + 128-bit integer
+ 32); // 64-bit pointer + padding + 128-bit integer
// Verify Wrapper function
Function *OutlinedFn =
diff --git a/mlir/test/Target/LLVMIR/openmp-llvm.mlir b/mlir/test/Target/LLVMIR/openmp-llvm.mlir
index e867dd8afcb9b..093790f7113e7 100644
--- a/mlir/test/Target/LLVMIR/openmp-llvm.mlir
+++ b/mlir/test/Target/LLVMIR/openmp-llvm.mlir
@@ -1897,7 +1897,7 @@ llvm.func @omp_atomic_capture_complex_swap(%x_arg: !llvm.ptr, %v_arg: !llvm.ptr)
// CHECK-LABEL: define void @omp_atomic_capture_complex8_swap
llvm.func @omp_atomic_capture_complex8_swap(%x_arg: !llvm.ptr, %v_arg: !llvm.ptr) {
- // CHECK: %[[ATOMIC_TEMP_LOAD:.*]] = alloca { double, double }, align 8
+ // CHECK: %[[ATOMIC_TEMP_LOAD:.*]] = alloca { double, double }, align 16
// CHECK: %[[X_NEW_VAL:.*]] = alloca { double, double }, align 8
// CHECK: call void @__atomic_load(i64 16, ptr %{{.*}}, ptr %[[ATOMIC_TEMP_LOAD]], i32 0)
// CHECK: %[[PHI:.*]] = phi { double, double }
diff --git a/polly/test/ScopInfo/multiple-types-non-power-of-two-2.ll b/polly/test/ScopInfo/multiple-types-non-power-of-two-2.ll
index 5890a5a2ea3bf..850884ccdb997 100644
--- a/polly/test/ScopInfo/multiple-types-non-power-of-two-2.ll
+++ b/polly/test/ScopInfo/multiple-types-non-power-of-two-2.ll
@@ -25,7 +25,7 @@
; CHECK: ReadAccess := [Reduction Type: NONE] [Scalar: 0]
; CHECK: { Stmt_bb2[i0] -> MemRef_A[o0] : 2i0 <= o0 <= 1 + 2i0 }
; CHECK: ReadAccess := [Reduction Type: NONE] [Scalar: 0]
-; CHECK: { Stmt_bb2[i0] -> MemRef_A[o0] : 3i0 <= o0 <= 2 + 3i0 }
+; CHECK: { Stmt_bb2[i0] -> MemRef_A[o0] : 3i0 <= o0 <= 3 + 3i0 }
; CHECK: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
; CHECK: { Stmt_bb2[i0] -> MemRef_A[o0] : 2i0 <= o0 <= 1 + 2i0 }
; CHECK: }
diff --git a/polly/test/ScopInfo/multiple-types-non-power-of-two.ll b/polly/test/ScopInfo/multiple-types-non-power-of-two.ll
index 3e8390aad300f..9b6dbb6d39d0d 100644
--- a/polly/test/ScopInfo/multiple-types-non-power-of-two.ll
+++ b/polly/test/ScopInfo/multiple-types-non-power-of-two.ll
@@ -21,7 +21,7 @@
; type i24 is modeled with allocation size i32 and that i40, i48 and i56 are
; modeled with allocation size i64. Larger types, e.g., i120, i192 and i248 are
; not rounded up to the next power-of-two allocation size, but rather to the
-; next multiple of 64.
+; next multiple of 128, the alignment of types larger than i64.
; The allocation size discussed above defines the number of canonical array
; elements accessed. For example, even though i24 only consists of 3 bytes,
@@ -52,7 +52,7 @@
; CHECK: ReadAccess := [Reduction Type: NONE] [Scalar: 0]
; CHECK: { Stmt_bb2[i0] -> MemRef_A[o0] : 16i0 <= o0 <= 15 + 16i0 };
; CHECK: ReadAccess := [Reduction Type: NONE] [Scalar: 0]
-; CHECK: { Stmt_bb2[i0] -> MemRef_A[o0] : 24i0 <= o0 <= 23 + 24i0 };
+; CHECK: { Stmt_bb2[i0] -> MemRef_A[o0] : 24i0 <= o0 <= 31 + 24i0 };
; CHECK: ReadAccess := [Reduction Type: NONE] [Scalar: 0]
; CHECK: { Stmt_bb2[i0] -> MemRef_A[o0] : 32i0 <= o0 <= 31 + 32i0 };
; CHECK: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
>From 3e63ec96d3bc68ed9e5025c68d18f5749214388d Mon Sep 17 00:00:00 2001
From: Simeon David Schaub <simeon at schaub.rocks>
Date: Sat, 8 Aug 2026 06:08:10 +0000
Subject: [PATCH 3/3] [clang][HIP] Test that AMDGPU kernarg layout matches
Clang's __int128 ABI
The kernarg segment offsets the backend emits are computed from the data
layout, while host code (and Clang's own record layout) uses the C-level
alignof(__int128) == 16. Pin both sides in one test: the static_asserts
fail if Clang's C layout changes, and the .offset/.size metadata checks
fail if the backend stops agreeing with it.
Co-Authored-By: Claude Fable 5 <noreply at anthropic.com>
---
.../amdgpu-kernarg-int128-layout.hip | 56 +++++++++++++++++++
1 file changed, 56 insertions(+)
create mode 100644 clang/test/CodeGenHIP/amdgpu-kernarg-int128-layout.hip
diff --git a/clang/test/CodeGenHIP/amdgpu-kernarg-int128-layout.hip b/clang/test/CodeGenHIP/amdgpu-kernarg-int128-layout.hip
new file mode 100644
index 0000000000000..08a3b8d49da35
--- /dev/null
+++ b/clang/test/CodeGenHIP/amdgpu-kernarg-int128-layout.hip
@@ -0,0 +1,56 @@
+// REQUIRES: amdgpu-registered-target
+// RUN: %clang_cc1 -fcuda-is-device -triple amdgcn-amd-amdhsa -target-cpu gfx900 -S -o - %s | FileCheck %s
+
+// Check that the kernarg segment layout the backend emits agrees with the
+// C-level layout Clang guarantees for __int128 (AMDGPUTargetInfo leaves
+// Int128Align at its 128-bit default). The static_asserts pin the frontend
+// side; the .offset/.size metadata entries pin the backend side. Before the
+// data layout gave i128 a default ABI alignment of 16 the backend placed
+// i128 kernel arguments at the next 8-byte slot, so host code computing
+// argument offsets with offsetof disagreed with the device-side layout.
+
+#define __global__ __attribute__((global))
+
+struct S {
+ char c;
+ __int128 x;
+};
+
+static_assert(__alignof__(__int128) == 16);
+static_assert(__builtin_offsetof(S, x) == 16);
+static_assert(sizeof(S) == 32);
+
+// CHECK-LABEL: {{^}}scalar_args:
+extern "C" __global__ void scalar_args(char c, __int128 x, int *out) {
+ *out = (int)(x >> 64) + c;
+}
+
+// CHECK-LABEL: {{^}}struct_arg:
+extern "C" __global__ void struct_arg(S s, int *out) {
+ *out = (int)(s.x >> 64) + s.c;
+}
+
+// The metadata block is emitted once at module scope, after all functions.
+// CHECK: .amdgpu_metadata
+
+// `x` must be aligned to 16 in the kernarg segment, not packed at 8, and the
+// following pointer argument starts after it at 32.
+// CHECK: .name: c
+// CHECK-NEXT: .offset: 0
+// CHECK-NEXT: .size: 1
+// CHECK: .name: x
+// CHECK-NEXT: .offset: 16
+// CHECK-NEXT: .size: 16
+// CHECK: .name: out
+// CHECK-NEXT: .offset: 32
+// CHECK-NEXT: .size: 8
+// CHECK: .name: scalar_args
+
+// The by-value struct occupies one 32-byte slot: field `x` lives at byte 16
+// of it, matching offsetof(S, x) above.
+// CHECK: .offset: 0
+// CHECK-NEXT: .size: 32
+// CHECK: .name: out
+// CHECK-NEXT: .offset: 32
+// CHECK-NEXT: .size: 8
+// CHECK: .name: struct_arg
More information about the llvm-commits
mailing list