[llvm] [SelectionDAG] Treat CopyFromReg as freezing the value (PR #85932)
Björn Pettersson via llvm-commits
llvm-commits at lists.llvm.org
Wed Mar 20 06:57:45 PDT 2024
https://github.com/bjope created https://github.com/llvm/llvm-project/pull/85932
The description of CopyFromReg in ISDOpcodes.h says that the input
valus is defined outside the scope of the current SelectionDAG. I
think that means that we basically can treat it as a FREEZE in the
sense that it can be seen as neither being undef nor poison.
Being able to fold freeze(CopyFromReg) into CopyFromReg seems
useful to avoid regressions if we start to introduce freeze
instruction in DAGCombiner/foldBoolSelectToLogic, e.g. to solve
https://github.com/llvm/llvm-project/issues/84653
Things _not_ dealt with in this patch:
- Depending on calling convention an input argument can be passed
also on the stack and not in a register. If it is allowed to treat
an argument received in a register as not being poison, then I think
we want to treat arguments received on the stack the same way. But
then we need to attribute load instructions, or add explicit FREEZE
when lowering formal arguments.
- A common pattern is that there is an AssertZext or AssertSext just
after CopyFromReg. I think that if we treat CopyFromReg as never
being poison, then it should be allowed to fold
(freeze(AssertZext(CopyFromReg))) -> AssertZext(CopyFromReg))
>From d170434d2e1072dc636daf0716d7aaa4dd816773 Mon Sep 17 00:00:00 2001
From: Bjorn Pettersson <bjorn.a.pettersson at ericsson.com>
Date: Wed, 13 Mar 2024 13:15:09 +0100
Subject: [PATCH 1/2] [SelectionDAG] Treat CopyFromReg as freezing the value
The description of CopyFromReg in ISDOpcodes.h says that the input
valus is defined outside the scope of the current SelectionDAG. I
think that means that we basically can treat it as a FREEZE in the
sense that it can be seen as neither being undef nor poison.
Being able to fold freeze(CopyFromReg) into CopyFromReg seems
useful to avoid regressions if we start to introduce freeze
instruction in DAGCombiner/foldBoolSelectToLogic, e.g. to solve
https://github.com/llvm/llvm-project/issues/84653
Things _not_ dealt with in this patch:
- Depending on calling convention an input argument can be passed
also on the stack and not in a register. If it is allowed to treat
an argument received in a register as not being poison, then I think
we want to treat arguments received on the stack the same way. But
then we need to attribute load instructions, or add explicit FREEZE
when lowering formal arguments.
- A common pattern is that there is an AssertZext or AssertSext just
after CopyFromReg. I think that if we treat CopyFromReg as never
being poison, then it should be allowed to fold
(freeze(AssertZext(CopyFromReg))) -> AssertZext(CopyFromReg))
---
llvm/include/llvm/CodeGen/ISDOpcodes.h | 1 +
llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp | 4 ++++
2 files changed, 5 insertions(+)
diff --git a/llvm/include/llvm/CodeGen/ISDOpcodes.h b/llvm/include/llvm/CodeGen/ISDOpcodes.h
index 49d51a27e3c0f6..800967cbfd2e7b 100644
--- a/llvm/include/llvm/CodeGen/ISDOpcodes.h
+++ b/llvm/include/llvm/CodeGen/ISDOpcodes.h
@@ -205,6 +205,7 @@ enum NodeType {
/// CopyFromReg - This node indicates that the input value is a virtual or
/// physical register that is defined outside of the scope of this
/// SelectionDAG. The register is available from the RegisterSDNode object.
+ /// Not that CopyFromReg is considered as also freezing the value.
CopyFromReg,
/// UNDEF - An undefined node.
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index 9d73a42df2a479..1996b998852e29 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -5012,6 +5012,10 @@ bool SelectionDAG::isGuaranteedNotToBeUndefOrPoison(SDValue Op,
case ISD::TargetFrameIndex:
return true;
+ case ISD::CopyFromReg:
+ // Treat CopyFromReg as freezing the value.
+ return true;
+
case ISD::UNDEF:
return PoisonOnly;
>From b78d4fe0ea2d5db52e69332942643a42dd3d4da3 Mon Sep 17 00:00:00 2001
From: Bjorn Pettersson <bjorn.a.pettersson at ericsson.com>
Date: Fri, 15 Mar 2024 15:57:54 +0100
Subject: [PATCH 2/2] to be squashed, update test checks
(need to verify that the diffs looks ok)
---
llvm/test/CodeGen/AArch64/combine-mul.ll | 7 +-
llvm/test/CodeGen/RISCV/alu64.ll | 2 +-
.../CodeGen/RISCV/atomicrmw-uinc-udec-wrap.ll | 16 +-
llvm/test/CodeGen/RISCV/double-convert.ll | 120 ++--
.../CodeGen/RISCV/double-round-conv-sat.ll | 18 +-
llvm/test/CodeGen/RISCV/float-convert.ll | 124 +++--
llvm/test/CodeGen/RISCV/forced-atomics.ll | 4 +-
llvm/test/CodeGen/RISCV/fpclamptosat.ll | 4 +-
llvm/test/CodeGen/RISCV/half-convert.ll | 32 +-
llvm/test/CodeGen/RISCV/iabs.ll | 4 +-
.../CodeGen/RISCV/rvv/fpclamptosat_vec.ll | 521 +++++++++---------
llvm/test/CodeGen/X86/abdu-vector-128.ll | 54 +-
.../CodeGen/X86/apx/kmov-postrapseudos.ll | 10 +-
.../CodeGen/X86/avx512-broadcast-arith.ll | 10 +-
llvm/test/CodeGen/X86/combine-mul.ll | 8 +-
.../X86/div-rem-pair-recomposition-signed.ll | 337 +++++------
llvm/test/CodeGen/X86/fold-masked-merge.ll | 4 +-
llvm/test/CodeGen/X86/freeze-binary.ll | 56 +-
llvm/test/CodeGen/X86/freeze-combine.ll | 16 +-
llvm/test/CodeGen/X86/gfni-funnel-shifts.ll | 4 +-
llvm/test/CodeGen/X86/gfni-rotates.ll | 14 +-
llvm/test/CodeGen/X86/known-never-zero.ll | 30 +-
llvm/test/CodeGen/X86/midpoint-int-vec-128.ll | 94 ++--
llvm/test/CodeGen/X86/midpoint-int-vec-256.ll | 32 +-
llvm/test/CodeGen/X86/pr38539.ll | 202 +++----
llvm/test/CodeGen/X86/pr62286.ll | 23 +-
.../CodeGen/X86/scheduler-backtracking.ll | 2 +-
.../test/CodeGen/X86/setcc-non-simple-type.ll | 78 ++-
llvm/test/CodeGen/X86/vec_saddo.ll | 14 +-
llvm/test/CodeGen/X86/vec_ssubo.ll | 14 +-
llvm/test/CodeGen/X86/vec_uaddo.ll | 14 +-
llvm/test/CodeGen/X86/vec_usubo.ll | 14 +-
llvm/test/CodeGen/X86/vector-bo-select.ll | 10 +-
llvm/test/CodeGen/X86/vector-fshr-128.ll | 86 +--
llvm/test/CodeGen/X86/vector-fshr-256.ll | 28 +-
llvm/test/CodeGen/X86/vector-fshr-sub128.ll | 12 +-
llvm/test/CodeGen/X86/vector-shift-shl-128.ll | 12 +-
37 files changed, 1019 insertions(+), 1011 deletions(-)
diff --git a/llvm/test/CodeGen/AArch64/combine-mul.ll b/llvm/test/CodeGen/AArch64/combine-mul.ll
index a2b0425308093d..c49e5ae6620a9e 100644
--- a/llvm/test/CodeGen/AArch64/combine-mul.ll
+++ b/llvm/test/CodeGen/AArch64/combine-mul.ll
@@ -44,8 +44,7 @@ define <4 x i1> @PR48683_vec_undef(<4 x i32> %x) {
define i64 @combine_mul_self_demandedbits(i64 %x) {
; CHECK-LABEL: combine_mul_self_demandedbits:
; CHECK: // %bb.0:
-; CHECK-NEXT: mul x8, x0, x0
-; CHECK-NEXT: and x0, x8, #0xfffffffffffffffd
+; CHECK-NEXT: mul x0, x0, x0
; CHECK-NEXT: ret
%1 = mul i64 %x, %x
%2 = and i64 %1, -3
@@ -77,7 +76,7 @@ define i8 @one_demanded_bit(i8 %x) {
define <2 x i64> @one_demanded_bit_splat(<2 x i64> %x) {
; CHECK-LABEL: one_demanded_bit_splat:
; CHECK: // %bb.0:
-; CHECK-NEXT: mov w8, #32
+; CHECK-NEXT: mov w8, #32 // =0x20
; CHECK-NEXT: shl v0.2d, v0.2d, #5
; CHECK-NEXT: dup v1.2d, x8
; CHECK-NEXT: and v0.16b, v0.16b, v1.16b
@@ -131,7 +130,7 @@ define i32 @squared_demanded_2_low_bits(i32 %x) {
define <2 x i64> @squared_demanded_2_low_bits_splat(<2 x i64> %x) {
; CHECK-LABEL: squared_demanded_2_low_bits_splat:
; CHECK: // %bb.0:
-; CHECK-NEXT: mov x8, #-2
+; CHECK-NEXT: mov x8, #-2 // =0xfffffffffffffffe
; CHECK-NEXT: dup v1.2d, x8
; CHECK-NEXT: orr v0.16b, v0.16b, v1.16b
; CHECK-NEXT: ret
diff --git a/llvm/test/CodeGen/RISCV/alu64.ll b/llvm/test/CodeGen/RISCV/alu64.ll
index d2ee80e6aa9513..f032756e007b68 100644
--- a/llvm/test/CodeGen/RISCV/alu64.ll
+++ b/llvm/test/CodeGen/RISCV/alu64.ll
@@ -57,8 +57,8 @@ define i64 @sltiu(i64 %a) nounwind {
;
; RV32I-LABEL: sltiu:
; RV32I: # %bb.0:
-; RV32I-NEXT: seqz a1, a1
; RV32I-NEXT: sltiu a0, a0, 3
+; RV32I-NEXT: seqz a1, a1
; RV32I-NEXT: and a0, a1, a0
; RV32I-NEXT: li a1, 0
; RV32I-NEXT: ret
diff --git a/llvm/test/CodeGen/RISCV/atomicrmw-uinc-udec-wrap.ll b/llvm/test/CodeGen/RISCV/atomicrmw-uinc-udec-wrap.ll
index 5914e45a153302..aa962d68fc5285 100644
--- a/llvm/test/CodeGen/RISCV/atomicrmw-uinc-udec-wrap.ll
+++ b/llvm/test/CodeGen/RISCV/atomicrmw-uinc-udec-wrap.ll
@@ -372,10 +372,10 @@ define i32 @atomicrmw_uinc_wrap_i32(ptr %ptr, i32 %val) {
; RV32IA-NEXT: # =>This Loop Header: Depth=1
; RV32IA-NEXT: # Child Loop BB2_3 Depth 2
; RV32IA-NEXT: mv a3, a2
-; RV32IA-NEXT: addi a4, a2, 1
-; RV32IA-NEXT: sltu a2, a2, a1
-; RV32IA-NEXT: neg a2, a2
-; RV32IA-NEXT: and a4, a2, a4
+; RV32IA-NEXT: addi a2, a2, 1
+; RV32IA-NEXT: sltu a4, a3, a1
+; RV32IA-NEXT: neg a4, a4
+; RV32IA-NEXT: and a4, a4, a2
; RV32IA-NEXT: .LBB2_3: # %atomicrmw.start
; RV32IA-NEXT: # Parent Loop BB2_1 Depth=1
; RV32IA-NEXT: # => This Inner Loop Header: Depth=2
@@ -607,10 +607,10 @@ define i64 @atomicrmw_uinc_wrap_i64(ptr %ptr, i64 %val) {
; RV64IA-NEXT: # =>This Loop Header: Depth=1
; RV64IA-NEXT: # Child Loop BB3_3 Depth 2
; RV64IA-NEXT: mv a3, a2
-; RV64IA-NEXT: addi a4, a2, 1
-; RV64IA-NEXT: sltu a2, a2, a1
-; RV64IA-NEXT: neg a2, a2
-; RV64IA-NEXT: and a4, a2, a4
+; RV64IA-NEXT: addi a2, a2, 1
+; RV64IA-NEXT: sltu a4, a3, a1
+; RV64IA-NEXT: neg a4, a4
+; RV64IA-NEXT: and a4, a4, a2
; RV64IA-NEXT: .LBB3_3: # %atomicrmw.start
; RV64IA-NEXT: # Parent Loop BB3_1 Depth=1
; RV64IA-NEXT: # => This Inner Loop Header: Depth=2
diff --git a/llvm/test/CodeGen/RISCV/double-convert.ll b/llvm/test/CodeGen/RISCV/double-convert.ll
index 3700a18bafc612..932682a89e28fb 100644
--- a/llvm/test/CodeGen/RISCV/double-convert.ll
+++ b/llvm/test/CodeGen/RISCV/double-convert.ll
@@ -771,9 +771,8 @@ define i64 @fcvt_l_d_sat(double %a) nounwind {
; RV32IFD-NEXT: feq.d a2, fs0, fs0
; RV32IFD-NEXT: neg a2, a2
; RV32IFD-NEXT: lui a4, 524288
-; RV32IFD-NEXT: li a5, 1
; RV32IFD-NEXT: lui a3, 524288
-; RV32IFD-NEXT: bne s2, a5, .LBB12_2
+; RV32IFD-NEXT: beqz s2, .LBB12_2
; RV32IFD-NEXT: # %bb.1: # %start
; RV32IFD-NEXT: mv a3, a1
; RV32IFD-NEXT: .LBB12_2: # %start
@@ -807,31 +806,29 @@ define i64 @fcvt_l_d_sat(double %a) nounwind {
; RV32IZFINXZDINX-NEXT: sw ra, 28(sp) # 4-byte Folded Spill
; RV32IZFINXZDINX-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; RV32IZFINXZDINX-NEXT: sw s1, 20(sp) # 4-byte Folded Spill
-; RV32IZFINXZDINX-NEXT: sw s2, 16(sp) # 4-byte Folded Spill
-; RV32IZFINXZDINX-NEXT: sw s3, 12(sp) # 4-byte Folded Spill
-; RV32IZFINXZDINX-NEXT: sw a0, 0(sp)
-; RV32IZFINXZDINX-NEXT: sw a1, 4(sp)
-; RV32IZFINXZDINX-NEXT: lw s0, 0(sp)
-; RV32IZFINXZDINX-NEXT: lw s1, 4(sp)
+; RV32IZFINXZDINX-NEXT: sw a0, 8(sp)
+; RV32IZFINXZDINX-NEXT: sw a1, 12(sp)
+; RV32IZFINXZDINX-NEXT: lw s0, 8(sp)
+; RV32IZFINXZDINX-NEXT: lw s1, 12(sp)
+; RV32IZFINXZDINX-NEXT: call __fixdfdi
; RV32IZFINXZDINX-NEXT: lui a2, %hi(.LCPI12_0)
; RV32IZFINXZDINX-NEXT: lw a3, %lo(.LCPI12_0+4)(a2)
; RV32IZFINXZDINX-NEXT: lw a2, %lo(.LCPI12_0)(a2)
-; RV32IZFINXZDINX-NEXT: fle.d s2, a2, s0
-; RV32IZFINXZDINX-NEXT: neg s3, s2
-; RV32IZFINXZDINX-NEXT: call __fixdfdi
-; RV32IZFINXZDINX-NEXT: lui a2, %hi(.LCPI12_1)
-; RV32IZFINXZDINX-NEXT: lw a3, %lo(.LCPI12_1+4)(a2)
-; RV32IZFINXZDINX-NEXT: lw a2, %lo(.LCPI12_1)(a2)
-; RV32IZFINXZDINX-NEXT: and a0, s3, a0
-; RV32IZFINXZDINX-NEXT: flt.d a3, a2, s0
+; RV32IZFINXZDINX-NEXT: lui a4, %hi(.LCPI12_1)
+; RV32IZFINXZDINX-NEXT: lw a5, %lo(.LCPI12_1+4)(a4)
+; RV32IZFINXZDINX-NEXT: lw a4, %lo(.LCPI12_1)(a4)
+; RV32IZFINXZDINX-NEXT: fle.d a6, a2, s0
+; RV32IZFINXZDINX-NEXT: neg a2, a6
+; RV32IZFINXZDINX-NEXT: and a0, a2, a0
+; RV32IZFINXZDINX-NEXT: flt.d a3, a4, s0
; RV32IZFINXZDINX-NEXT: neg a2, a3
; RV32IZFINXZDINX-NEXT: or a0, a2, a0
; RV32IZFINXZDINX-NEXT: feq.d a2, s0, s0
; RV32IZFINXZDINX-NEXT: neg a2, a2
; RV32IZFINXZDINX-NEXT: lui a5, 524288
-; RV32IZFINXZDINX-NEXT: li a6, 1
+; RV32IZFINXZDINX-NEXT: li a7, 1
; RV32IZFINXZDINX-NEXT: lui a4, 524288
-; RV32IZFINXZDINX-NEXT: bne s2, a6, .LBB12_2
+; RV32IZFINXZDINX-NEXT: bne a6, a7, .LBB12_2
; RV32IZFINXZDINX-NEXT: # %bb.1: # %start
; RV32IZFINXZDINX-NEXT: mv a4, a1
; RV32IZFINXZDINX-NEXT: .LBB12_2: # %start
@@ -844,8 +841,6 @@ define i64 @fcvt_l_d_sat(double %a) nounwind {
; RV32IZFINXZDINX-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; RV32IZFINXZDINX-NEXT: lw s0, 24(sp) # 4-byte Folded Reload
; RV32IZFINXZDINX-NEXT: lw s1, 20(sp) # 4-byte Folded Reload
-; RV32IZFINXZDINX-NEXT: lw s2, 16(sp) # 4-byte Folded Reload
-; RV32IZFINXZDINX-NEXT: lw s3, 12(sp) # 4-byte Folded Reload
; RV32IZFINXZDINX-NEXT: addi sp, sp, 32
; RV32IZFINXZDINX-NEXT: ret
;
@@ -868,33 +863,32 @@ define i64 @fcvt_l_d_sat(double %a) nounwind {
; RV32I-NEXT: sw s3, 12(sp) # 4-byte Folded Spill
; RV32I-NEXT: sw s4, 8(sp) # 4-byte Folded Spill
; RV32I-NEXT: sw s5, 4(sp) # 4-byte Folded Spill
-; RV32I-NEXT: sw s6, 0(sp) # 4-byte Folded Spill
; RV32I-NEXT: mv s0, a1
; RV32I-NEXT: mv s1, a0
+; RV32I-NEXT: lui a3, 278016
+; RV32I-NEXT: addi a3, a3, -1
+; RV32I-NEXT: li a2, -1
+; RV32I-NEXT: call __gtdf2
+; RV32I-NEXT: mv s2, a0
; RV32I-NEXT: lui a3, 802304
+; RV32I-NEXT: mv a0, s1
+; RV32I-NEXT: mv a1, s0
; RV32I-NEXT: li a2, 0
; RV32I-NEXT: call __gedf2
-; RV32I-NEXT: mv s2, a0
+; RV32I-NEXT: mv s3, a0
; RV32I-NEXT: mv a0, s1
; RV32I-NEXT: mv a1, s0
; RV32I-NEXT: call __fixdfdi
-; RV32I-NEXT: mv s3, a0
-; RV32I-NEXT: mv s4, a1
-; RV32I-NEXT: lui s6, 524288
-; RV32I-NEXT: bgez s2, .LBB12_2
+; RV32I-NEXT: mv s4, a0
+; RV32I-NEXT: mv s5, a1
+; RV32I-NEXT: lui a0, 524288
+; RV32I-NEXT: bgez s3, .LBB12_2
; RV32I-NEXT: # %bb.1: # %start
-; RV32I-NEXT: lui s4, 524288
+; RV32I-NEXT: lui s5, 524288
; RV32I-NEXT: .LBB12_2: # %start
-; RV32I-NEXT: lui a3, 278016
-; RV32I-NEXT: addi a3, a3, -1
-; RV32I-NEXT: li a2, -1
-; RV32I-NEXT: mv a0, s1
-; RV32I-NEXT: mv a1, s0
-; RV32I-NEXT: call __gtdf2
-; RV32I-NEXT: mv s5, a0
-; RV32I-NEXT: blez a0, .LBB12_4
+; RV32I-NEXT: blez s2, .LBB12_4
; RV32I-NEXT: # %bb.3: # %start
-; RV32I-NEXT: addi s4, s6, -1
+; RV32I-NEXT: addi s5, a0, -1
; RV32I-NEXT: .LBB12_4: # %start
; RV32I-NEXT: mv a0, s1
; RV32I-NEXT: mv a1, s0
@@ -903,11 +897,11 @@ define i64 @fcvt_l_d_sat(double %a) nounwind {
; RV32I-NEXT: call __unorddf2
; RV32I-NEXT: snez a0, a0
; RV32I-NEXT: addi a0, a0, -1
-; RV32I-NEXT: and a1, a0, s4
-; RV32I-NEXT: slti a2, s2, 0
+; RV32I-NEXT: and a1, a0, s5
+; RV32I-NEXT: slti a2, s3, 0
; RV32I-NEXT: addi a2, a2, -1
-; RV32I-NEXT: and a2, a2, s3
-; RV32I-NEXT: sgtz a3, s5
+; RV32I-NEXT: and a2, a2, s4
+; RV32I-NEXT: sgtz a3, s2
; RV32I-NEXT: neg a3, a3
; RV32I-NEXT: or a2, a3, a2
; RV32I-NEXT: and a0, a0, a2
@@ -918,7 +912,6 @@ define i64 @fcvt_l_d_sat(double %a) nounwind {
; RV32I-NEXT: lw s3, 12(sp) # 4-byte Folded Reload
; RV32I-NEXT: lw s4, 8(sp) # 4-byte Folded Reload
; RV32I-NEXT: lw s5, 4(sp) # 4-byte Folded Reload
-; RV32I-NEXT: lw s6, 0(sp) # 4-byte Folded Reload
; RV32I-NEXT: addi sp, sp, 32
; RV32I-NEXT: ret
;
@@ -1027,22 +1020,23 @@ define i64 @fcvt_lu_d_sat(double %a) nounwind {
; RV32IFD-NEXT: addi sp, sp, -16
; RV32IFD-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
; RV32IFD-NEXT: sw s0, 8(sp) # 4-byte Folded Spill
-; RV32IFD-NEXT: sw s1, 4(sp) # 4-byte Folded Spill
-; RV32IFD-NEXT: lui a0, %hi(.LCPI14_0)
-; RV32IFD-NEXT: fld fa5, %lo(.LCPI14_0)(a0)
-; RV32IFD-NEXT: flt.d a0, fa5, fa0
-; RV32IFD-NEXT: neg s0, a0
+; RV32IFD-NEXT: fsd fs0, 0(sp) # 8-byte Folded Spill
+; RV32IFD-NEXT: fmv.d fs0, fa0
; RV32IFD-NEXT: fcvt.d.w fa5, zero
; RV32IFD-NEXT: fle.d a0, fa5, fa0
-; RV32IFD-NEXT: neg s1, a0
+; RV32IFD-NEXT: neg s0, a0
; RV32IFD-NEXT: call __fixunsdfdi
-; RV32IFD-NEXT: and a0, s1, a0
-; RV32IFD-NEXT: or a0, s0, a0
-; RV32IFD-NEXT: and a1, s1, a1
-; RV32IFD-NEXT: or a1, s0, a1
+; RV32IFD-NEXT: lui a2, %hi(.LCPI14_0)
+; RV32IFD-NEXT: fld fa5, %lo(.LCPI14_0)(a2)
+; RV32IFD-NEXT: and a0, s0, a0
+; RV32IFD-NEXT: flt.d a2, fa5, fs0
+; RV32IFD-NEXT: neg a2, a2
+; RV32IFD-NEXT: or a0, a2, a0
+; RV32IFD-NEXT: and a1, s0, a1
+; RV32IFD-NEXT: or a1, a2, a1
; RV32IFD-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
; RV32IFD-NEXT: lw s0, 8(sp) # 4-byte Folded Reload
-; RV32IFD-NEXT: lw s1, 4(sp) # 4-byte Folded Reload
+; RV32IFD-NEXT: fld fs0, 0(sp) # 8-byte Folded Reload
; RV32IFD-NEXT: addi sp, sp, 16
; RV32IFD-NEXT: ret
;
@@ -1061,28 +1055,26 @@ define i64 @fcvt_lu_d_sat(double %a) nounwind {
; RV32IZFINXZDINX-NEXT: sw ra, 28(sp) # 4-byte Folded Spill
; RV32IZFINXZDINX-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; RV32IZFINXZDINX-NEXT: sw s1, 20(sp) # 4-byte Folded Spill
-; RV32IZFINXZDINX-NEXT: sw s2, 16(sp) # 4-byte Folded Spill
; RV32IZFINXZDINX-NEXT: sw a0, 8(sp)
; RV32IZFINXZDINX-NEXT: sw a1, 12(sp)
; RV32IZFINXZDINX-NEXT: lw s0, 8(sp)
; RV32IZFINXZDINX-NEXT: lw s1, 12(sp)
+; RV32IZFINXZDINX-NEXT: call __fixunsdfdi
; RV32IZFINXZDINX-NEXT: fcvt.d.w a2, zero
+; RV32IZFINXZDINX-NEXT: lui a4, %hi(.LCPI14_0)
+; RV32IZFINXZDINX-NEXT: lw a5, %lo(.LCPI14_0+4)(a4)
+; RV32IZFINXZDINX-NEXT: lw a4, %lo(.LCPI14_0)(a4)
; RV32IZFINXZDINX-NEXT: fle.d a2, a2, s0
-; RV32IZFINXZDINX-NEXT: neg s2, a2
-; RV32IZFINXZDINX-NEXT: call __fixunsdfdi
-; RV32IZFINXZDINX-NEXT: lui a2, %hi(.LCPI14_0)
-; RV32IZFINXZDINX-NEXT: lw a3, %lo(.LCPI14_0+4)(a2)
-; RV32IZFINXZDINX-NEXT: lw a2, %lo(.LCPI14_0)(a2)
-; RV32IZFINXZDINX-NEXT: and a0, s2, a0
-; RV32IZFINXZDINX-NEXT: flt.d a2, a2, s0
; RV32IZFINXZDINX-NEXT: neg a2, a2
-; RV32IZFINXZDINX-NEXT: or a0, a2, a0
-; RV32IZFINXZDINX-NEXT: and a1, s2, a1
-; RV32IZFINXZDINX-NEXT: or a1, a2, a1
+; RV32IZFINXZDINX-NEXT: and a0, a2, a0
+; RV32IZFINXZDINX-NEXT: flt.d a3, a4, s0
+; RV32IZFINXZDINX-NEXT: neg a3, a3
+; RV32IZFINXZDINX-NEXT: or a0, a3, a0
+; RV32IZFINXZDINX-NEXT: and a1, a2, a1
+; RV32IZFINXZDINX-NEXT: or a1, a3, a1
; RV32IZFINXZDINX-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; RV32IZFINXZDINX-NEXT: lw s0, 24(sp) # 4-byte Folded Reload
; RV32IZFINXZDINX-NEXT: lw s1, 20(sp) # 4-byte Folded Reload
-; RV32IZFINXZDINX-NEXT: lw s2, 16(sp) # 4-byte Folded Reload
; RV32IZFINXZDINX-NEXT: addi sp, sp, 32
; RV32IZFINXZDINX-NEXT: ret
;
diff --git a/llvm/test/CodeGen/RISCV/double-round-conv-sat.ll b/llvm/test/CodeGen/RISCV/double-round-conv-sat.ll
index 7cdf18e2fea9c0..a3a35ba1ccc8d9 100644
--- a/llvm/test/CodeGen/RISCV/double-round-conv-sat.ll
+++ b/llvm/test/CodeGen/RISCV/double-round-conv-sat.ll
@@ -73,9 +73,8 @@ define i64 @test_floor_si64(double %x) nounwind {
; RV32IFD-NEXT: feq.d a2, fs0, fs0
; RV32IFD-NEXT: neg a2, a2
; RV32IFD-NEXT: lui a4, 524288
-; RV32IFD-NEXT: li a5, 1
; RV32IFD-NEXT: lui a3, 524288
-; RV32IFD-NEXT: bne s2, a5, .LBB1_2
+; RV32IFD-NEXT: beqz s2, .LBB1_2
; RV32IFD-NEXT: # %bb.1:
; RV32IFD-NEXT: mv a3, a1
; RV32IFD-NEXT: .LBB1_2:
@@ -353,9 +352,8 @@ define i64 @test_ceil_si64(double %x) nounwind {
; RV32IFD-NEXT: feq.d a2, fs0, fs0
; RV32IFD-NEXT: neg a2, a2
; RV32IFD-NEXT: lui a4, 524288
-; RV32IFD-NEXT: li a5, 1
; RV32IFD-NEXT: lui a3, 524288
-; RV32IFD-NEXT: bne s2, a5, .LBB5_2
+; RV32IFD-NEXT: beqz s2, .LBB5_2
; RV32IFD-NEXT: # %bb.1:
; RV32IFD-NEXT: mv a3, a1
; RV32IFD-NEXT: .LBB5_2:
@@ -633,9 +631,8 @@ define i64 @test_trunc_si64(double %x) nounwind {
; RV32IFD-NEXT: feq.d a2, fs0, fs0
; RV32IFD-NEXT: neg a2, a2
; RV32IFD-NEXT: lui a4, 524288
-; RV32IFD-NEXT: li a5, 1
; RV32IFD-NEXT: lui a3, 524288
-; RV32IFD-NEXT: bne s2, a5, .LBB9_2
+; RV32IFD-NEXT: beqz s2, .LBB9_2
; RV32IFD-NEXT: # %bb.1:
; RV32IFD-NEXT: mv a3, a1
; RV32IFD-NEXT: .LBB9_2:
@@ -913,9 +910,8 @@ define i64 @test_round_si64(double %x) nounwind {
; RV32IFD-NEXT: feq.d a2, fs0, fs0
; RV32IFD-NEXT: neg a2, a2
; RV32IFD-NEXT: lui a4, 524288
-; RV32IFD-NEXT: li a5, 1
; RV32IFD-NEXT: lui a3, 524288
-; RV32IFD-NEXT: bne s2, a5, .LBB13_2
+; RV32IFD-NEXT: beqz s2, .LBB13_2
; RV32IFD-NEXT: # %bb.1:
; RV32IFD-NEXT: mv a3, a1
; RV32IFD-NEXT: .LBB13_2:
@@ -1193,9 +1189,8 @@ define i64 @test_roundeven_si64(double %x) nounwind {
; RV32IFD-NEXT: feq.d a2, fs0, fs0
; RV32IFD-NEXT: neg a2, a2
; RV32IFD-NEXT: lui a4, 524288
-; RV32IFD-NEXT: li a5, 1
; RV32IFD-NEXT: lui a3, 524288
-; RV32IFD-NEXT: bne s2, a5, .LBB17_2
+; RV32IFD-NEXT: beqz s2, .LBB17_2
; RV32IFD-NEXT: # %bb.1:
; RV32IFD-NEXT: mv a3, a1
; RV32IFD-NEXT: .LBB17_2:
@@ -1473,9 +1468,8 @@ define i64 @test_rint_si64(double %x) nounwind {
; RV32IFD-NEXT: feq.d a2, fs0, fs0
; RV32IFD-NEXT: neg a2, a2
; RV32IFD-NEXT: lui a4, 524288
-; RV32IFD-NEXT: li a5, 1
; RV32IFD-NEXT: lui a3, 524288
-; RV32IFD-NEXT: bne s2, a5, .LBB21_2
+; RV32IFD-NEXT: beqz s2, .LBB21_2
; RV32IFD-NEXT: # %bb.1:
; RV32IFD-NEXT: mv a3, a1
; RV32IFD-NEXT: .LBB21_2:
diff --git a/llvm/test/CodeGen/RISCV/float-convert.ll b/llvm/test/CodeGen/RISCV/float-convert.ll
index 9fb78d4c4d5210..ee54b45afa4357 100644
--- a/llvm/test/CodeGen/RISCV/float-convert.ll
+++ b/llvm/test/CodeGen/RISCV/float-convert.ll
@@ -275,26 +275,24 @@ define i32 @fcvt_wu_s_sat(float %a) nounwind {
; RV32I-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
; RV32I-NEXT: sw s0, 8(sp) # 4-byte Folded Spill
; RV32I-NEXT: sw s1, 4(sp) # 4-byte Folded Spill
-; RV32I-NEXT: sw s2, 0(sp) # 4-byte Folded Spill
; RV32I-NEXT: mv s0, a0
-; RV32I-NEXT: lui a1, 325632
-; RV32I-NEXT: addi a1, a1, -1
-; RV32I-NEXT: call __gtsf2
-; RV32I-NEXT: sgtz a0, a0
-; RV32I-NEXT: neg s1, a0
-; RV32I-NEXT: mv a0, s0
; RV32I-NEXT: li a1, 0
; RV32I-NEXT: call __gesf2
; RV32I-NEXT: slti a0, a0, 0
-; RV32I-NEXT: addi s2, a0, -1
+; RV32I-NEXT: addi s1, a0, -1
; RV32I-NEXT: mv a0, s0
; RV32I-NEXT: call __fixunssfsi
-; RV32I-NEXT: and a0, s2, a0
-; RV32I-NEXT: or a0, s1, a0
+; RV32I-NEXT: and s1, s1, a0
+; RV32I-NEXT: lui a1, 325632
+; RV32I-NEXT: addi a1, a1, -1
+; RV32I-NEXT: mv a0, s0
+; RV32I-NEXT: call __gtsf2
+; RV32I-NEXT: sgtz a0, a0
+; RV32I-NEXT: neg a0, a0
+; RV32I-NEXT: or a0, a0, s1
; RV32I-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
; RV32I-NEXT: lw s0, 8(sp) # 4-byte Folded Reload
; RV32I-NEXT: lw s1, 4(sp) # 4-byte Folded Reload
-; RV32I-NEXT: lw s2, 0(sp) # 4-byte Folded Reload
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
@@ -869,22 +867,23 @@ define i64 @fcvt_lu_s_sat(float %a) nounwind {
; RV32IF-NEXT: addi sp, sp, -16
; RV32IF-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
; RV32IF-NEXT: sw s0, 8(sp) # 4-byte Folded Spill
-; RV32IF-NEXT: sw s1, 4(sp) # 4-byte Folded Spill
-; RV32IF-NEXT: lui a0, %hi(.LCPI14_0)
-; RV32IF-NEXT: flw fa5, %lo(.LCPI14_0)(a0)
-; RV32IF-NEXT: flt.s a0, fa5, fa0
-; RV32IF-NEXT: neg s0, a0
+; RV32IF-NEXT: fsw fs0, 4(sp) # 4-byte Folded Spill
+; RV32IF-NEXT: fmv.s fs0, fa0
; RV32IF-NEXT: fmv.w.x fa5, zero
; RV32IF-NEXT: fle.s a0, fa5, fa0
-; RV32IF-NEXT: neg s1, a0
+; RV32IF-NEXT: neg s0, a0
; RV32IF-NEXT: call __fixunssfdi
-; RV32IF-NEXT: and a0, s1, a0
-; RV32IF-NEXT: or a0, s0, a0
-; RV32IF-NEXT: and a1, s1, a1
-; RV32IF-NEXT: or a1, s0, a1
+; RV32IF-NEXT: lui a2, %hi(.LCPI14_0)
+; RV32IF-NEXT: flw fa5, %lo(.LCPI14_0)(a2)
+; RV32IF-NEXT: and a0, s0, a0
+; RV32IF-NEXT: flt.s a2, fa5, fs0
+; RV32IF-NEXT: neg a2, a2
+; RV32IF-NEXT: or a0, a2, a0
+; RV32IF-NEXT: and a1, s0, a1
+; RV32IF-NEXT: or a1, a2, a1
; RV32IF-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
; RV32IF-NEXT: lw s0, 8(sp) # 4-byte Folded Reload
-; RV32IF-NEXT: lw s1, 4(sp) # 4-byte Folded Reload
+; RV32IF-NEXT: flw fs0, 4(sp) # 4-byte Folded Reload
; RV32IF-NEXT: addi sp, sp, 16
; RV32IF-NEXT: ret
;
@@ -903,17 +902,19 @@ define i64 @fcvt_lu_s_sat(float %a) nounwind {
; RV32IZFINX-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
; RV32IZFINX-NEXT: sw s0, 8(sp) # 4-byte Folded Spill
; RV32IZFINX-NEXT: sw s1, 4(sp) # 4-byte Folded Spill
-; RV32IZFINX-NEXT: lui a1, %hi(.LCPI14_0)
-; RV32IZFINX-NEXT: lw a1, %lo(.LCPI14_0)(a1)
-; RV32IZFINX-NEXT: flt.s a1, a1, a0
-; RV32IZFINX-NEXT: neg s0, a1
-; RV32IZFINX-NEXT: fle.s a1, zero, a0
-; RV32IZFINX-NEXT: neg s1, a1
+; RV32IZFINX-NEXT: mv s0, a0
+; RV32IZFINX-NEXT: fle.s a0, zero, a0
+; RV32IZFINX-NEXT: neg s1, a0
+; RV32IZFINX-NEXT: mv a0, s0
; RV32IZFINX-NEXT: call __fixunssfdi
+; RV32IZFINX-NEXT: lui a2, %hi(.LCPI14_0)
+; RV32IZFINX-NEXT: lw a2, %lo(.LCPI14_0)(a2)
; RV32IZFINX-NEXT: and a0, s1, a0
-; RV32IZFINX-NEXT: or a0, s0, a0
+; RV32IZFINX-NEXT: flt.s a2, a2, s0
+; RV32IZFINX-NEXT: neg a2, a2
+; RV32IZFINX-NEXT: or a0, a2, a0
; RV32IZFINX-NEXT: and a1, s1, a1
-; RV32IZFINX-NEXT: or a1, s0, a1
+; RV32IZFINX-NEXT: or a1, a2, a1
; RV32IZFINX-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
; RV32IZFINX-NEXT: lw s0, 8(sp) # 4-byte Folded Reload
; RV32IZFINX-NEXT: lw s1, 4(sp) # 4-byte Folded Reload
@@ -931,33 +932,36 @@ define i64 @fcvt_lu_s_sat(float %a) nounwind {
;
; RV32I-LABEL: fcvt_lu_s_sat:
; RV32I: # %bb.0: # %start
-; RV32I-NEXT: addi sp, sp, -16
-; RV32I-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
-; RV32I-NEXT: sw s0, 8(sp) # 4-byte Folded Spill
-; RV32I-NEXT: sw s1, 4(sp) # 4-byte Folded Spill
-; RV32I-NEXT: sw s2, 0(sp) # 4-byte Folded Spill
+; RV32I-NEXT: addi sp, sp, -32
+; RV32I-NEXT: sw ra, 28(sp) # 4-byte Folded Spill
+; RV32I-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
+; RV32I-NEXT: sw s1, 20(sp) # 4-byte Folded Spill
+; RV32I-NEXT: sw s2, 16(sp) # 4-byte Folded Spill
+; RV32I-NEXT: sw s3, 12(sp) # 4-byte Folded Spill
; RV32I-NEXT: mv s0, a0
-; RV32I-NEXT: lui a1, 391168
-; RV32I-NEXT: addi a1, a1, -1
-; RV32I-NEXT: call __gtsf2
-; RV32I-NEXT: sgtz a0, a0
-; RV32I-NEXT: neg s1, a0
-; RV32I-NEXT: mv a0, s0
; RV32I-NEXT: li a1, 0
; RV32I-NEXT: call __gesf2
; RV32I-NEXT: slti a0, a0, 0
; RV32I-NEXT: addi s2, a0, -1
; RV32I-NEXT: mv a0, s0
; RV32I-NEXT: call __fixunssfdi
-; RV32I-NEXT: and a0, s2, a0
-; RV32I-NEXT: or a0, s1, a0
-; RV32I-NEXT: and a1, s2, a1
-; RV32I-NEXT: or a1, s1, a1
-; RV32I-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
-; RV32I-NEXT: lw s0, 8(sp) # 4-byte Folded Reload
-; RV32I-NEXT: lw s1, 4(sp) # 4-byte Folded Reload
-; RV32I-NEXT: lw s2, 0(sp) # 4-byte Folded Reload
-; RV32I-NEXT: addi sp, sp, 16
+; RV32I-NEXT: mv s1, a1
+; RV32I-NEXT: and s3, s2, a0
+; RV32I-NEXT: lui a1, 391168
+; RV32I-NEXT: addi a1, a1, -1
+; RV32I-NEXT: mv a0, s0
+; RV32I-NEXT: call __gtsf2
+; RV32I-NEXT: sgtz a0, a0
+; RV32I-NEXT: neg a1, a0
+; RV32I-NEXT: or a0, a1, s3
+; RV32I-NEXT: and a2, s2, s1
+; RV32I-NEXT: or a1, a1, a2
+; RV32I-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
+; RV32I-NEXT: lw s0, 24(sp) # 4-byte Folded Reload
+; RV32I-NEXT: lw s1, 20(sp) # 4-byte Folded Reload
+; RV32I-NEXT: lw s2, 16(sp) # 4-byte Folded Reload
+; RV32I-NEXT: lw s3, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT: addi sp, sp, 32
; RV32I-NEXT: ret
;
; RV64I-LABEL: fcvt_lu_s_sat:
@@ -2091,26 +2095,24 @@ define zeroext i32 @fcvt_wu_s_sat_zext(float %a) nounwind {
; RV32I-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
; RV32I-NEXT: sw s0, 8(sp) # 4-byte Folded Spill
; RV32I-NEXT: sw s1, 4(sp) # 4-byte Folded Spill
-; RV32I-NEXT: sw s2, 0(sp) # 4-byte Folded Spill
; RV32I-NEXT: mv s0, a0
-; RV32I-NEXT: lui a1, 325632
-; RV32I-NEXT: addi a1, a1, -1
-; RV32I-NEXT: call __gtsf2
-; RV32I-NEXT: sgtz a0, a0
-; RV32I-NEXT: neg s1, a0
-; RV32I-NEXT: mv a0, s0
; RV32I-NEXT: li a1, 0
; RV32I-NEXT: call __gesf2
; RV32I-NEXT: slti a0, a0, 0
-; RV32I-NEXT: addi s2, a0, -1
+; RV32I-NEXT: addi s1, a0, -1
; RV32I-NEXT: mv a0, s0
; RV32I-NEXT: call __fixunssfsi
-; RV32I-NEXT: and a0, s2, a0
-; RV32I-NEXT: or a0, s1, a0
+; RV32I-NEXT: and s1, s1, a0
+; RV32I-NEXT: lui a1, 325632
+; RV32I-NEXT: addi a1, a1, -1
+; RV32I-NEXT: mv a0, s0
+; RV32I-NEXT: call __gtsf2
+; RV32I-NEXT: sgtz a0, a0
+; RV32I-NEXT: neg a0, a0
+; RV32I-NEXT: or a0, a0, s1
; RV32I-NEXT: lw ra, 12(sp) # 4-byte Folded Reload
; RV32I-NEXT: lw s0, 8(sp) # 4-byte Folded Reload
; RV32I-NEXT: lw s1, 4(sp) # 4-byte Folded Reload
-; RV32I-NEXT: lw s2, 0(sp) # 4-byte Folded Reload
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
diff --git a/llvm/test/CodeGen/RISCV/forced-atomics.ll b/llvm/test/CodeGen/RISCV/forced-atomics.ll
index c303690aadfff8..f6a53a9d76dd35 100644
--- a/llvm/test/CodeGen/RISCV/forced-atomics.ll
+++ b/llvm/test/CodeGen/RISCV/forced-atomics.ll
@@ -3567,8 +3567,8 @@ define i64 @rmw64_umax_seq_cst(ptr %p) nounwind {
; RV32-NEXT: # in Loop: Header=BB51_2 Depth=1
; RV32-NEXT: neg a3, a0
; RV32-NEXT: and a3, a3, a1
-; RV32-NEXT: sw a1, 4(sp)
; RV32-NEXT: sw a4, 0(sp)
+; RV32-NEXT: sw a1, 4(sp)
; RV32-NEXT: mv a1, sp
; RV32-NEXT: li a4, 5
; RV32-NEXT: li a5, 5
@@ -3659,8 +3659,8 @@ define i64 @rmw64_umin_seq_cst(ptr %p) nounwind {
; RV32-NEXT: # in Loop: Header=BB52_2 Depth=1
; RV32-NEXT: neg a3, a0
; RV32-NEXT: and a3, a3, a1
-; RV32-NEXT: sw a1, 4(sp)
; RV32-NEXT: sw a4, 0(sp)
+; RV32-NEXT: sw a1, 4(sp)
; RV32-NEXT: mv a1, sp
; RV32-NEXT: li a4, 5
; RV32-NEXT: li a5, 5
diff --git a/llvm/test/CodeGen/RISCV/fpclamptosat.ll b/llvm/test/CodeGen/RISCV/fpclamptosat.ll
index 06ab813faf0253..deb5a6d4013d49 100644
--- a/llvm/test/CodeGen/RISCV/fpclamptosat.ll
+++ b/llvm/test/CodeGen/RISCV/fpclamptosat.ll
@@ -114,8 +114,8 @@ define i32 @utest_f64i32(double %x) {
; RV32IF-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
; RV32IF-NEXT: .cfi_offset ra, -4
; RV32IF-NEXT: call __fixunsdfdi
-; RV32IF-NEXT: seqz a1, a1
; RV32IF-NEXT: sltiu a2, a0, -1
+; RV32IF-NEXT: seqz a1, a1
; RV32IF-NEXT: and a1, a1, a2
; RV32IF-NEXT: addi a1, a1, -1
; RV32IF-NEXT: or a0, a1, a0
@@ -429,8 +429,8 @@ define i32 @utesth_f16i32(half %x) {
; RV32-NEXT: .cfi_offset ra, -4
; RV32-NEXT: call __extendhfsf2
; RV32-NEXT: call __fixunssfdi
-; RV32-NEXT: seqz a1, a1
; RV32-NEXT: sltiu a2, a0, -1
+; RV32-NEXT: seqz a1, a1
; RV32-NEXT: and a1, a1, a2
; RV32-NEXT: addi a1, a1, -1
; RV32-NEXT: or a0, a1, a0
diff --git a/llvm/test/CodeGen/RISCV/half-convert.ll b/llvm/test/CodeGen/RISCV/half-convert.ll
index 28ac6e272e11df..6c326e74c536b4 100644
--- a/llvm/test/CodeGen/RISCV/half-convert.ll
+++ b/llvm/test/CodeGen/RISCV/half-convert.ll
@@ -2524,42 +2524,38 @@ define i64 @fcvt_l_h_sat(half %a) nounwind {
; RV32ID-NEXT: sw ra, 28(sp) # 4-byte Folded Spill
; RV32ID-NEXT: sw s0, 24(sp) # 4-byte Folded Spill
; RV32ID-NEXT: sw s1, 20(sp) # 4-byte Folded Spill
-; RV32ID-NEXT: sw s2, 16(sp) # 4-byte Folded Spill
-; RV32ID-NEXT: sw s3, 12(sp) # 4-byte Folded Spill
-; RV32ID-NEXT: fsd fs0, 0(sp) # 8-byte Folded Spill
+; RV32ID-NEXT: fsd fs0, 8(sp) # 8-byte Folded Spill
; RV32ID-NEXT: call __extendhfsf2
-; RV32ID-NEXT: lui a0, %hi(.LCPI10_0)
-; RV32ID-NEXT: flw fa5, %lo(.LCPI10_0)(a0)
; RV32ID-NEXT: fmv.s fs0, fa0
-; RV32ID-NEXT: flt.s s0, fa5, fa0
-; RV32ID-NEXT: neg s1, s0
; RV32ID-NEXT: lui a0, 913408
; RV32ID-NEXT: fmv.w.x fa5, a0
-; RV32ID-NEXT: fle.s s2, fa5, fa0
-; RV32ID-NEXT: neg s3, s2
+; RV32ID-NEXT: fle.s s0, fa5, fa0
+; RV32ID-NEXT: neg s1, s0
; RV32ID-NEXT: call __fixsfdi
-; RV32ID-NEXT: and a0, s3, a0
-; RV32ID-NEXT: or a0, s1, a0
+; RV32ID-NEXT: lui a2, %hi(.LCPI10_0)
+; RV32ID-NEXT: flw fa5, %lo(.LCPI10_0)(a2)
+; RV32ID-NEXT: and a0, s1, a0
+; RV32ID-NEXT: flt.s a4, fa5, fs0
+; RV32ID-NEXT: neg a2, a4
+; RV32ID-NEXT: or a0, a2, a0
; RV32ID-NEXT: feq.s a2, fs0, fs0
; RV32ID-NEXT: neg a2, a2
-; RV32ID-NEXT: lui a4, 524288
+; RV32ID-NEXT: lui a5, 524288
; RV32ID-NEXT: lui a3, 524288
-; RV32ID-NEXT: beqz s2, .LBB10_2
+; RV32ID-NEXT: beqz s0, .LBB10_2
; RV32ID-NEXT: # %bb.1: # %start
; RV32ID-NEXT: mv a3, a1
; RV32ID-NEXT: .LBB10_2: # %start
; RV32ID-NEXT: and a0, a2, a0
-; RV32ID-NEXT: beqz s0, .LBB10_4
+; RV32ID-NEXT: beqz a4, .LBB10_4
; RV32ID-NEXT: # %bb.3:
-; RV32ID-NEXT: addi a3, a4, -1
+; RV32ID-NEXT: addi a3, a5, -1
; RV32ID-NEXT: .LBB10_4: # %start
; RV32ID-NEXT: and a1, a2, a3
; RV32ID-NEXT: lw ra, 28(sp) # 4-byte Folded Reload
; RV32ID-NEXT: lw s0, 24(sp) # 4-byte Folded Reload
; RV32ID-NEXT: lw s1, 20(sp) # 4-byte Folded Reload
-; RV32ID-NEXT: lw s2, 16(sp) # 4-byte Folded Reload
-; RV32ID-NEXT: lw s3, 12(sp) # 4-byte Folded Reload
-; RV32ID-NEXT: fld fs0, 0(sp) # 8-byte Folded Reload
+; RV32ID-NEXT: fld fs0, 8(sp) # 8-byte Folded Reload
; RV32ID-NEXT: addi sp, sp, 32
; RV32ID-NEXT: ret
;
diff --git a/llvm/test/CodeGen/RISCV/iabs.ll b/llvm/test/CodeGen/RISCV/iabs.ll
index 98c886333d69a0..a0c85ab4dca7f7 100644
--- a/llvm/test/CodeGen/RISCV/iabs.ll
+++ b/llvm/test/CodeGen/RISCV/iabs.ll
@@ -630,8 +630,8 @@ define void @zext16_abs8(i8 %x, ptr %p) {
; RV32I-LABEL: zext16_abs8:
; RV32I: # %bb.0:
; RV32I-NEXT: slli a0, a0, 24
-; RV32I-NEXT: srai a0, a0, 24
; RV32I-NEXT: srai a2, a0, 31
+; RV32I-NEXT: srai a0, a0, 24
; RV32I-NEXT: xor a0, a0, a2
; RV32I-NEXT: sub a0, a0, a2
; RV32I-NEXT: sh a0, 0(a1)
@@ -648,8 +648,8 @@ define void @zext16_abs8(i8 %x, ptr %p) {
; RV64I-LABEL: zext16_abs8:
; RV64I: # %bb.0:
; RV64I-NEXT: slli a0, a0, 56
-; RV64I-NEXT: srai a0, a0, 56
; RV64I-NEXT: srai a2, a0, 63
+; RV64I-NEXT: srai a0, a0, 56
; RV64I-NEXT: xor a0, a0, a2
; RV64I-NEXT: subw a0, a0, a2
; RV64I-NEXT: sh a0, 0(a1)
diff --git a/llvm/test/CodeGen/RISCV/rvv/fpclamptosat_vec.ll b/llvm/test/CodeGen/RISCV/rvv/fpclamptosat_vec.ll
index eb7894ede04647..74f61708faaca1 100644
--- a/llvm/test/CodeGen/RISCV/rvv/fpclamptosat_vec.ll
+++ b/llvm/test/CodeGen/RISCV/rvv/fpclamptosat_vec.ll
@@ -2106,65 +2106,66 @@ define <2 x i64> @stest_f64i64(<2 x double> %x) {
; CHECK-NOV-NEXT: .cfi_offset s0, -16
; CHECK-NOV-NEXT: .cfi_offset s1, -24
; CHECK-NOV-NEXT: .cfi_offset fs0, -32
-; CHECK-NOV-NEXT: fmv.d fs0, fa0
-; CHECK-NOV-NEXT: fmv.d fa0, fa1
+; CHECK-NOV-NEXT: fmv.d fs0, fa1
; CHECK-NOV-NEXT: call __fixdfti
; CHECK-NOV-NEXT: mv s0, a0
; CHECK-NOV-NEXT: mv s1, a1
; CHECK-NOV-NEXT: fmv.d fa0, fs0
; CHECK-NOV-NEXT: call __fixdfti
-; CHECK-NOV-NEXT: li a2, -1
-; CHECK-NOV-NEXT: srli a3, a2, 1
-; CHECK-NOV-NEXT: beqz s1, .LBB18_3
+; CHECK-NOV-NEXT: mv a2, a0
+; CHECK-NOV-NEXT: li a0, -1
+; CHECK-NOV-NEXT: srli a3, a0, 1
+; CHECK-NOV-NEXT: beqz a1, .LBB18_3
; CHECK-NOV-NEXT: # %bb.1: # %entry
-; CHECK-NOV-NEXT: slti a4, s1, 0
-; CHECK-NOV-NEXT: bnez a1, .LBB18_4
+; CHECK-NOV-NEXT: slti a4, a1, 0
+; CHECK-NOV-NEXT: bnez s1, .LBB18_4
; CHECK-NOV-NEXT: .LBB18_2:
-; CHECK-NOV-NEXT: sltu a5, a0, a3
+; CHECK-NOV-NEXT: sltu a5, s0, a3
; CHECK-NOV-NEXT: beqz a5, .LBB18_5
; CHECK-NOV-NEXT: j .LBB18_6
; CHECK-NOV-NEXT: .LBB18_3:
-; CHECK-NOV-NEXT: sltu a4, s0, a3
-; CHECK-NOV-NEXT: beqz a1, .LBB18_2
+; CHECK-NOV-NEXT: sltu a4, a2, a3
+; CHECK-NOV-NEXT: beqz s1, .LBB18_2
; CHECK-NOV-NEXT: .LBB18_4: # %entry
-; CHECK-NOV-NEXT: slti a5, a1, 0
+; CHECK-NOV-NEXT: slti a5, s1, 0
; CHECK-NOV-NEXT: bnez a5, .LBB18_6
; CHECK-NOV-NEXT: .LBB18_5: # %entry
-; CHECK-NOV-NEXT: mv a0, a3
+; CHECK-NOV-NEXT: mv s0, a3
; CHECK-NOV-NEXT: .LBB18_6: # %entry
; CHECK-NOV-NEXT: neg a6, a5
; CHECK-NOV-NEXT: neg a5, a4
-; CHECK-NOV-NEXT: and a5, a5, s1
+; CHECK-NOV-NEXT: and a5, a5, a1
; CHECK-NOV-NEXT: bnez a4, .LBB18_8
; CHECK-NOV-NEXT: # %bb.7: # %entry
-; CHECK-NOV-NEXT: mv s0, a3
+; CHECK-NOV-NEXT: mv a2, a3
; CHECK-NOV-NEXT: .LBB18_8: # %entry
-; CHECK-NOV-NEXT: and a4, a6, a1
-; CHECK-NOV-NEXT: slli a1, a2, 63
-; CHECK-NOV-NEXT: beq a5, a2, .LBB18_11
+; CHECK-NOV-NEXT: and a4, a6, s1
+; CHECK-NOV-NEXT: slli a1, a0, 63
+; CHECK-NOV-NEXT: beq a5, a0, .LBB18_11
; CHECK-NOV-NEXT: # %bb.9: # %entry
; CHECK-NOV-NEXT: slti a3, a5, 0
; CHECK-NOV-NEXT: xori a3, a3, 1
-; CHECK-NOV-NEXT: bne a4, a2, .LBB18_12
+; CHECK-NOV-NEXT: bne a4, a0, .LBB18_12
; CHECK-NOV-NEXT: .LBB18_10:
-; CHECK-NOV-NEXT: sltu a2, a1, a0
-; CHECK-NOV-NEXT: beqz a2, .LBB18_13
+; CHECK-NOV-NEXT: sltu a0, a1, s0
+; CHECK-NOV-NEXT: beqz a0, .LBB18_13
; CHECK-NOV-NEXT: j .LBB18_14
; CHECK-NOV-NEXT: .LBB18_11:
-; CHECK-NOV-NEXT: sltu a3, a1, s0
-; CHECK-NOV-NEXT: beq a4, a2, .LBB18_10
+; CHECK-NOV-NEXT: sltu a3, a1, a2
+; CHECK-NOV-NEXT: beq a4, a0, .LBB18_10
; CHECK-NOV-NEXT: .LBB18_12: # %entry
-; CHECK-NOV-NEXT: slti a2, a4, 0
-; CHECK-NOV-NEXT: xori a2, a2, 1
-; CHECK-NOV-NEXT: bnez a2, .LBB18_14
+; CHECK-NOV-NEXT: slti a0, a4, 0
+; CHECK-NOV-NEXT: xori a0, a0, 1
+; CHECK-NOV-NEXT: bnez a0, .LBB18_14
; CHECK-NOV-NEXT: .LBB18_13: # %entry
-; CHECK-NOV-NEXT: mv a0, a1
+; CHECK-NOV-NEXT: mv s0, a1
; CHECK-NOV-NEXT: .LBB18_14: # %entry
; CHECK-NOV-NEXT: bnez a3, .LBB18_16
; CHECK-NOV-NEXT: # %bb.15: # %entry
-; CHECK-NOV-NEXT: mv s0, a1
+; CHECK-NOV-NEXT: mv a2, a1
; CHECK-NOV-NEXT: .LBB18_16: # %entry
-; CHECK-NOV-NEXT: mv a1, s0
+; CHECK-NOV-NEXT: mv a0, s0
+; CHECK-NOV-NEXT: mv a1, a2
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s1, 8(sp) # 8-byte Folded Reload
@@ -2189,43 +2190,43 @@ define <2 x i64> @stest_f64i64(<2 x double> %x) {
; CHECK-V-NEXT: addi a0, sp, 32
; CHECK-V-NEXT: vs1r.v v8, (a0) # Unknown-size Folded Spill
; CHECK-V-NEXT: vsetivli zero, 1, e64, m1, ta, ma
-; CHECK-V-NEXT: vfmv.f.s fa0, v8
+; CHECK-V-NEXT: vslidedown.vi v9, v8, 1
+; CHECK-V-NEXT: vfmv.f.s fa0, v9
; CHECK-V-NEXT: call __fixdfti
; CHECK-V-NEXT: mv s0, a0
; CHECK-V-NEXT: mv s1, a1
; CHECK-V-NEXT: vsetivli zero, 1, e64, m1, ta, ma
; CHECK-V-NEXT: addi a0, sp, 32
; CHECK-V-NEXT: vl1r.v v8, (a0) # Unknown-size Folded Reload
-; CHECK-V-NEXT: vslidedown.vi v8, v8, 1
; CHECK-V-NEXT: vfmv.f.s fa0, v8
; CHECK-V-NEXT: call __fixdfti
; CHECK-V-NEXT: li a2, -1
; CHECK-V-NEXT: srli a3, a2, 1
-; CHECK-V-NEXT: beqz s1, .LBB18_3
+; CHECK-V-NEXT: beqz a1, .LBB18_3
; CHECK-V-NEXT: # %bb.1: # %entry
-; CHECK-V-NEXT: slti a4, s1, 0
-; CHECK-V-NEXT: bnez a1, .LBB18_4
+; CHECK-V-NEXT: slti a4, a1, 0
+; CHECK-V-NEXT: bnez s1, .LBB18_4
; CHECK-V-NEXT: .LBB18_2:
-; CHECK-V-NEXT: sltu a5, a0, a3
+; CHECK-V-NEXT: sltu a5, s0, a3
; CHECK-V-NEXT: beqz a5, .LBB18_5
; CHECK-V-NEXT: j .LBB18_6
; CHECK-V-NEXT: .LBB18_3:
-; CHECK-V-NEXT: sltu a4, s0, a3
-; CHECK-V-NEXT: beqz a1, .LBB18_2
+; CHECK-V-NEXT: sltu a4, a0, a3
+; CHECK-V-NEXT: beqz s1, .LBB18_2
; CHECK-V-NEXT: .LBB18_4: # %entry
-; CHECK-V-NEXT: slti a5, a1, 0
+; CHECK-V-NEXT: slti a5, s1, 0
; CHECK-V-NEXT: bnez a5, .LBB18_6
; CHECK-V-NEXT: .LBB18_5: # %entry
-; CHECK-V-NEXT: mv a0, a3
+; CHECK-V-NEXT: mv s0, a3
; CHECK-V-NEXT: .LBB18_6: # %entry
; CHECK-V-NEXT: neg a6, a5
; CHECK-V-NEXT: neg a5, a4
-; CHECK-V-NEXT: and a5, a5, s1
+; CHECK-V-NEXT: and a5, a5, a1
; CHECK-V-NEXT: bnez a4, .LBB18_8
; CHECK-V-NEXT: # %bb.7: # %entry
-; CHECK-V-NEXT: mv s0, a3
+; CHECK-V-NEXT: mv a0, a3
; CHECK-V-NEXT: .LBB18_8: # %entry
-; CHECK-V-NEXT: and a4, a6, a1
+; CHECK-V-NEXT: and a4, a6, s1
; CHECK-V-NEXT: slli a1, a2, 63
; CHECK-V-NEXT: beq a5, a2, .LBB18_11
; CHECK-V-NEXT: # %bb.9: # %entry
@@ -2233,26 +2234,26 @@ define <2 x i64> @stest_f64i64(<2 x double> %x) {
; CHECK-V-NEXT: xori a3, a3, 1
; CHECK-V-NEXT: bne a4, a2, .LBB18_12
; CHECK-V-NEXT: .LBB18_10:
-; CHECK-V-NEXT: sltu a2, a1, a0
+; CHECK-V-NEXT: sltu a2, a1, s0
; CHECK-V-NEXT: beqz a2, .LBB18_13
; CHECK-V-NEXT: j .LBB18_14
; CHECK-V-NEXT: .LBB18_11:
-; CHECK-V-NEXT: sltu a3, a1, s0
+; CHECK-V-NEXT: sltu a3, a1, a0
; CHECK-V-NEXT: beq a4, a2, .LBB18_10
; CHECK-V-NEXT: .LBB18_12: # %entry
; CHECK-V-NEXT: slti a2, a4, 0
; CHECK-V-NEXT: xori a2, a2, 1
; CHECK-V-NEXT: bnez a2, .LBB18_14
; CHECK-V-NEXT: .LBB18_13: # %entry
-; CHECK-V-NEXT: mv a0, a1
+; CHECK-V-NEXT: mv s0, a1
; CHECK-V-NEXT: .LBB18_14: # %entry
; CHECK-V-NEXT: bnez a3, .LBB18_16
; CHECK-V-NEXT: # %bb.15: # %entry
-; CHECK-V-NEXT: mv s0, a1
+; CHECK-V-NEXT: mv a0, a1
; CHECK-V-NEXT: .LBB18_16: # %entry
; CHECK-V-NEXT: vsetivli zero, 2, e64, m1, ta, ma
-; CHECK-V-NEXT: vmv.s.x v8, s0
-; CHECK-V-NEXT: vmv.s.x v9, a0
+; CHECK-V-NEXT: vmv.s.x v8, a0
+; CHECK-V-NEXT: vmv.s.x v9, s0
; CHECK-V-NEXT: vslideup.vi v8, v9, 1
; CHECK-V-NEXT: csrr a0, vlenb
; CHECK-V-NEXT: slli a0, a0, 1
@@ -2285,19 +2286,19 @@ define <2 x i64> @utest_f64i64(<2 x double> %x) {
; CHECK-NOV-NEXT: .cfi_offset s0, -16
; CHECK-NOV-NEXT: .cfi_offset s1, -24
; CHECK-NOV-NEXT: .cfi_offset fs0, -32
-; CHECK-NOV-NEXT: fmv.d fs0, fa0
-; CHECK-NOV-NEXT: fmv.d fa0, fa1
+; CHECK-NOV-NEXT: fmv.d fs0, fa1
; CHECK-NOV-NEXT: call __fixunsdfti
; CHECK-NOV-NEXT: mv s0, a0
; CHECK-NOV-NEXT: mv s1, a1
; CHECK-NOV-NEXT: fmv.d fa0, fs0
; CHECK-NOV-NEXT: call __fixunsdfti
-; CHECK-NOV-NEXT: snez a2, s1
; CHECK-NOV-NEXT: snez a1, a1
+; CHECK-NOV-NEXT: snez a2, s1
+; CHECK-NOV-NEXT: addi a2, a2, -1
+; CHECK-NOV-NEXT: and a2, a2, s0
; CHECK-NOV-NEXT: addi a1, a1, -1
-; CHECK-NOV-NEXT: and a0, a1, a0
-; CHECK-NOV-NEXT: addi a1, a2, -1
-; CHECK-NOV-NEXT: and a1, a1, s0
+; CHECK-NOV-NEXT: and a1, a1, a0
+; CHECK-NOV-NEXT: mv a0, a2
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s1, 8(sp) # 8-byte Folded Reload
@@ -2322,25 +2323,25 @@ define <2 x i64> @utest_f64i64(<2 x double> %x) {
; CHECK-V-NEXT: addi a0, sp, 32
; CHECK-V-NEXT: vs1r.v v8, (a0) # Unknown-size Folded Spill
; CHECK-V-NEXT: vsetivli zero, 1, e64, m1, ta, ma
-; CHECK-V-NEXT: vfmv.f.s fa0, v8
+; CHECK-V-NEXT: vslidedown.vi v9, v8, 1
+; CHECK-V-NEXT: vfmv.f.s fa0, v9
; CHECK-V-NEXT: call __fixunsdfti
; CHECK-V-NEXT: mv s0, a0
; CHECK-V-NEXT: mv s1, a1
; CHECK-V-NEXT: vsetivli zero, 1, e64, m1, ta, ma
; CHECK-V-NEXT: addi a0, sp, 32
; CHECK-V-NEXT: vl1r.v v8, (a0) # Unknown-size Folded Reload
-; CHECK-V-NEXT: vslidedown.vi v8, v8, 1
; CHECK-V-NEXT: vfmv.f.s fa0, v8
; CHECK-V-NEXT: call __fixunsdfti
-; CHECK-V-NEXT: snez a2, s1
; CHECK-V-NEXT: snez a1, a1
-; CHECK-V-NEXT: addi a1, a1, -1
-; CHECK-V-NEXT: and a0, a1, a0
+; CHECK-V-NEXT: snez a2, s1
; CHECK-V-NEXT: addi a2, a2, -1
; CHECK-V-NEXT: and a2, a2, s0
+; CHECK-V-NEXT: addi a1, a1, -1
+; CHECK-V-NEXT: and a0, a1, a0
; CHECK-V-NEXT: vsetivli zero, 2, e64, m1, ta, ma
-; CHECK-V-NEXT: vmv.s.x v8, a2
-; CHECK-V-NEXT: vmv.s.x v9, a0
+; CHECK-V-NEXT: vmv.s.x v8, a0
+; CHECK-V-NEXT: vmv.s.x v9, a2
; CHECK-V-NEXT: vslideup.vi v8, v9, 1
; CHECK-V-NEXT: csrr a0, vlenb
; CHECK-V-NEXT: slli a0, a0, 1
@@ -2382,32 +2383,32 @@ define <2 x i64> @ustest_f64i64(<2 x double> %x) {
; CHECK-NOV-NEXT: # %bb.1: # %entry
; CHECK-NOV-NEXT: li a2, 1
; CHECK-NOV-NEXT: .LBB20_2: # %entry
-; CHECK-NOV-NEXT: slti a4, s1, 1
; CHECK-NOV-NEXT: slti a3, a1, 1
+; CHECK-NOV-NEXT: slti a4, s1, 1
; CHECK-NOV-NEXT: blez a1, .LBB20_4
; CHECK-NOV-NEXT: # %bb.3: # %entry
; CHECK-NOV-NEXT: li a1, 1
; CHECK-NOV-NEXT: .LBB20_4: # %entry
+; CHECK-NOV-NEXT: neg a4, a4
; CHECK-NOV-NEXT: neg a3, a3
; CHECK-NOV-NEXT: and a3, a3, a0
-; CHECK-NOV-NEXT: neg a0, a4
; CHECK-NOV-NEXT: beqz a1, .LBB20_7
; CHECK-NOV-NEXT: # %bb.5: # %entry
; CHECK-NOV-NEXT: sgtz a1, a1
-; CHECK-NOV-NEXT: and a0, a0, s0
+; CHECK-NOV-NEXT: and a4, a4, s0
; CHECK-NOV-NEXT: bnez a2, .LBB20_8
; CHECK-NOV-NEXT: .LBB20_6:
-; CHECK-NOV-NEXT: snez a2, a0
+; CHECK-NOV-NEXT: snez a0, a4
; CHECK-NOV-NEXT: j .LBB20_9
; CHECK-NOV-NEXT: .LBB20_7:
; CHECK-NOV-NEXT: snez a1, a3
-; CHECK-NOV-NEXT: and a0, a0, s0
+; CHECK-NOV-NEXT: and a4, a4, s0
; CHECK-NOV-NEXT: beqz a2, .LBB20_6
; CHECK-NOV-NEXT: .LBB20_8: # %entry
-; CHECK-NOV-NEXT: sgtz a2, a2
+; CHECK-NOV-NEXT: sgtz a0, a2
; CHECK-NOV-NEXT: .LBB20_9: # %entry
-; CHECK-NOV-NEXT: neg a2, a2
-; CHECK-NOV-NEXT: and a0, a2, a0
+; CHECK-NOV-NEXT: neg a0, a0
+; CHECK-NOV-NEXT: and a0, a0, a4
; CHECK-NOV-NEXT: neg a1, a1
; CHECK-NOV-NEXT: and a1, a1, a3
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
@@ -2449,15 +2450,15 @@ define <2 x i64> @ustest_f64i64(<2 x double> %x) {
; CHECK-V-NEXT: # %bb.1: # %entry
; CHECK-V-NEXT: li a2, 1
; CHECK-V-NEXT: .LBB20_2: # %entry
-; CHECK-V-NEXT: slti a3, s1, 1
; CHECK-V-NEXT: slti a4, a1, 1
+; CHECK-V-NEXT: slti a3, s1, 1
; CHECK-V-NEXT: blez a1, .LBB20_4
; CHECK-V-NEXT: # %bb.3: # %entry
; CHECK-V-NEXT: li a1, 1
; CHECK-V-NEXT: .LBB20_4: # %entry
+; CHECK-V-NEXT: neg a3, a3
; CHECK-V-NEXT: neg a4, a4
; CHECK-V-NEXT: and a0, a4, a0
-; CHECK-V-NEXT: neg a3, a3
; CHECK-V-NEXT: beqz a1, .LBB20_7
; CHECK-V-NEXT: # %bb.5: # %entry
; CHECK-V-NEXT: sgtz a1, a1
@@ -2512,65 +2513,66 @@ define <2 x i64> @stest_f32i64(<2 x float> %x) {
; CHECK-NOV-NEXT: .cfi_offset s0, -16
; CHECK-NOV-NEXT: .cfi_offset s1, -24
; CHECK-NOV-NEXT: .cfi_offset fs0, -32
-; CHECK-NOV-NEXT: fmv.s fs0, fa0
-; CHECK-NOV-NEXT: fmv.s fa0, fa1
+; CHECK-NOV-NEXT: fmv.s fs0, fa1
; CHECK-NOV-NEXT: call __fixsfti
; CHECK-NOV-NEXT: mv s0, a0
; CHECK-NOV-NEXT: mv s1, a1
; CHECK-NOV-NEXT: fmv.s fa0, fs0
; CHECK-NOV-NEXT: call __fixsfti
-; CHECK-NOV-NEXT: li a2, -1
-; CHECK-NOV-NEXT: srli a3, a2, 1
-; CHECK-NOV-NEXT: beqz s1, .LBB21_3
+; CHECK-NOV-NEXT: mv a2, a0
+; CHECK-NOV-NEXT: li a0, -1
+; CHECK-NOV-NEXT: srli a3, a0, 1
+; CHECK-NOV-NEXT: beqz a1, .LBB21_3
; CHECK-NOV-NEXT: # %bb.1: # %entry
-; CHECK-NOV-NEXT: slti a4, s1, 0
-; CHECK-NOV-NEXT: bnez a1, .LBB21_4
+; CHECK-NOV-NEXT: slti a4, a1, 0
+; CHECK-NOV-NEXT: bnez s1, .LBB21_4
; CHECK-NOV-NEXT: .LBB21_2:
-; CHECK-NOV-NEXT: sltu a5, a0, a3
+; CHECK-NOV-NEXT: sltu a5, s0, a3
; CHECK-NOV-NEXT: beqz a5, .LBB21_5
; CHECK-NOV-NEXT: j .LBB21_6
; CHECK-NOV-NEXT: .LBB21_3:
-; CHECK-NOV-NEXT: sltu a4, s0, a3
-; CHECK-NOV-NEXT: beqz a1, .LBB21_2
+; CHECK-NOV-NEXT: sltu a4, a2, a3
+; CHECK-NOV-NEXT: beqz s1, .LBB21_2
; CHECK-NOV-NEXT: .LBB21_4: # %entry
-; CHECK-NOV-NEXT: slti a5, a1, 0
+; CHECK-NOV-NEXT: slti a5, s1, 0
; CHECK-NOV-NEXT: bnez a5, .LBB21_6
; CHECK-NOV-NEXT: .LBB21_5: # %entry
-; CHECK-NOV-NEXT: mv a0, a3
+; CHECK-NOV-NEXT: mv s0, a3
; CHECK-NOV-NEXT: .LBB21_6: # %entry
; CHECK-NOV-NEXT: neg a6, a5
; CHECK-NOV-NEXT: neg a5, a4
-; CHECK-NOV-NEXT: and a5, a5, s1
+; CHECK-NOV-NEXT: and a5, a5, a1
; CHECK-NOV-NEXT: bnez a4, .LBB21_8
; CHECK-NOV-NEXT: # %bb.7: # %entry
-; CHECK-NOV-NEXT: mv s0, a3
+; CHECK-NOV-NEXT: mv a2, a3
; CHECK-NOV-NEXT: .LBB21_8: # %entry
-; CHECK-NOV-NEXT: and a4, a6, a1
-; CHECK-NOV-NEXT: slli a1, a2, 63
-; CHECK-NOV-NEXT: beq a5, a2, .LBB21_11
+; CHECK-NOV-NEXT: and a4, a6, s1
+; CHECK-NOV-NEXT: slli a1, a0, 63
+; CHECK-NOV-NEXT: beq a5, a0, .LBB21_11
; CHECK-NOV-NEXT: # %bb.9: # %entry
; CHECK-NOV-NEXT: slti a3, a5, 0
; CHECK-NOV-NEXT: xori a3, a3, 1
-; CHECK-NOV-NEXT: bne a4, a2, .LBB21_12
+; CHECK-NOV-NEXT: bne a4, a0, .LBB21_12
; CHECK-NOV-NEXT: .LBB21_10:
-; CHECK-NOV-NEXT: sltu a2, a1, a0
-; CHECK-NOV-NEXT: beqz a2, .LBB21_13
+; CHECK-NOV-NEXT: sltu a0, a1, s0
+; CHECK-NOV-NEXT: beqz a0, .LBB21_13
; CHECK-NOV-NEXT: j .LBB21_14
; CHECK-NOV-NEXT: .LBB21_11:
-; CHECK-NOV-NEXT: sltu a3, a1, s0
-; CHECK-NOV-NEXT: beq a4, a2, .LBB21_10
+; CHECK-NOV-NEXT: sltu a3, a1, a2
+; CHECK-NOV-NEXT: beq a4, a0, .LBB21_10
; CHECK-NOV-NEXT: .LBB21_12: # %entry
-; CHECK-NOV-NEXT: slti a2, a4, 0
-; CHECK-NOV-NEXT: xori a2, a2, 1
-; CHECK-NOV-NEXT: bnez a2, .LBB21_14
+; CHECK-NOV-NEXT: slti a0, a4, 0
+; CHECK-NOV-NEXT: xori a0, a0, 1
+; CHECK-NOV-NEXT: bnez a0, .LBB21_14
; CHECK-NOV-NEXT: .LBB21_13: # %entry
-; CHECK-NOV-NEXT: mv a0, a1
+; CHECK-NOV-NEXT: mv s0, a1
; CHECK-NOV-NEXT: .LBB21_14: # %entry
; CHECK-NOV-NEXT: bnez a3, .LBB21_16
; CHECK-NOV-NEXT: # %bb.15: # %entry
-; CHECK-NOV-NEXT: mv s0, a1
+; CHECK-NOV-NEXT: mv a2, a1
; CHECK-NOV-NEXT: .LBB21_16: # %entry
-; CHECK-NOV-NEXT: mv a1, s0
+; CHECK-NOV-NEXT: mv a0, s0
+; CHECK-NOV-NEXT: mv a1, a2
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s1, 8(sp) # 8-byte Folded Reload
@@ -2595,43 +2597,43 @@ define <2 x i64> @stest_f32i64(<2 x float> %x) {
; CHECK-V-NEXT: addi a0, sp, 32
; CHECK-V-NEXT: vs1r.v v8, (a0) # Unknown-size Folded Spill
; CHECK-V-NEXT: vsetivli zero, 1, e32, mf2, ta, ma
-; CHECK-V-NEXT: vfmv.f.s fa0, v8
+; CHECK-V-NEXT: vslidedown.vi v9, v8, 1
+; CHECK-V-NEXT: vfmv.f.s fa0, v9
; CHECK-V-NEXT: call __fixsfti
; CHECK-V-NEXT: mv s0, a0
; CHECK-V-NEXT: mv s1, a1
; CHECK-V-NEXT: vsetivli zero, 1, e32, mf2, ta, ma
; CHECK-V-NEXT: addi a0, sp, 32
; CHECK-V-NEXT: vl1r.v v8, (a0) # Unknown-size Folded Reload
-; CHECK-V-NEXT: vslidedown.vi v8, v8, 1
; CHECK-V-NEXT: vfmv.f.s fa0, v8
; CHECK-V-NEXT: call __fixsfti
; CHECK-V-NEXT: li a2, -1
; CHECK-V-NEXT: srli a3, a2, 1
-; CHECK-V-NEXT: beqz s1, .LBB21_3
+; CHECK-V-NEXT: beqz a1, .LBB21_3
; CHECK-V-NEXT: # %bb.1: # %entry
-; CHECK-V-NEXT: slti a4, s1, 0
-; CHECK-V-NEXT: bnez a1, .LBB21_4
+; CHECK-V-NEXT: slti a4, a1, 0
+; CHECK-V-NEXT: bnez s1, .LBB21_4
; CHECK-V-NEXT: .LBB21_2:
-; CHECK-V-NEXT: sltu a5, a0, a3
+; CHECK-V-NEXT: sltu a5, s0, a3
; CHECK-V-NEXT: beqz a5, .LBB21_5
; CHECK-V-NEXT: j .LBB21_6
; CHECK-V-NEXT: .LBB21_3:
-; CHECK-V-NEXT: sltu a4, s0, a3
-; CHECK-V-NEXT: beqz a1, .LBB21_2
+; CHECK-V-NEXT: sltu a4, a0, a3
+; CHECK-V-NEXT: beqz s1, .LBB21_2
; CHECK-V-NEXT: .LBB21_4: # %entry
-; CHECK-V-NEXT: slti a5, a1, 0
+; CHECK-V-NEXT: slti a5, s1, 0
; CHECK-V-NEXT: bnez a5, .LBB21_6
; CHECK-V-NEXT: .LBB21_5: # %entry
-; CHECK-V-NEXT: mv a0, a3
+; CHECK-V-NEXT: mv s0, a3
; CHECK-V-NEXT: .LBB21_6: # %entry
; CHECK-V-NEXT: neg a6, a5
; CHECK-V-NEXT: neg a5, a4
-; CHECK-V-NEXT: and a5, a5, s1
+; CHECK-V-NEXT: and a5, a5, a1
; CHECK-V-NEXT: bnez a4, .LBB21_8
; CHECK-V-NEXT: # %bb.7: # %entry
-; CHECK-V-NEXT: mv s0, a3
+; CHECK-V-NEXT: mv a0, a3
; CHECK-V-NEXT: .LBB21_8: # %entry
-; CHECK-V-NEXT: and a4, a6, a1
+; CHECK-V-NEXT: and a4, a6, s1
; CHECK-V-NEXT: slli a1, a2, 63
; CHECK-V-NEXT: beq a5, a2, .LBB21_11
; CHECK-V-NEXT: # %bb.9: # %entry
@@ -2639,26 +2641,26 @@ define <2 x i64> @stest_f32i64(<2 x float> %x) {
; CHECK-V-NEXT: xori a3, a3, 1
; CHECK-V-NEXT: bne a4, a2, .LBB21_12
; CHECK-V-NEXT: .LBB21_10:
-; CHECK-V-NEXT: sltu a2, a1, a0
+; CHECK-V-NEXT: sltu a2, a1, s0
; CHECK-V-NEXT: beqz a2, .LBB21_13
; CHECK-V-NEXT: j .LBB21_14
; CHECK-V-NEXT: .LBB21_11:
-; CHECK-V-NEXT: sltu a3, a1, s0
+; CHECK-V-NEXT: sltu a3, a1, a0
; CHECK-V-NEXT: beq a4, a2, .LBB21_10
; CHECK-V-NEXT: .LBB21_12: # %entry
; CHECK-V-NEXT: slti a2, a4, 0
; CHECK-V-NEXT: xori a2, a2, 1
; CHECK-V-NEXT: bnez a2, .LBB21_14
; CHECK-V-NEXT: .LBB21_13: # %entry
-; CHECK-V-NEXT: mv a0, a1
+; CHECK-V-NEXT: mv s0, a1
; CHECK-V-NEXT: .LBB21_14: # %entry
; CHECK-V-NEXT: bnez a3, .LBB21_16
; CHECK-V-NEXT: # %bb.15: # %entry
-; CHECK-V-NEXT: mv s0, a1
+; CHECK-V-NEXT: mv a0, a1
; CHECK-V-NEXT: .LBB21_16: # %entry
; CHECK-V-NEXT: vsetivli zero, 2, e64, m1, ta, ma
-; CHECK-V-NEXT: vmv.s.x v8, s0
-; CHECK-V-NEXT: vmv.s.x v9, a0
+; CHECK-V-NEXT: vmv.s.x v8, a0
+; CHECK-V-NEXT: vmv.s.x v9, s0
; CHECK-V-NEXT: vslideup.vi v8, v9, 1
; CHECK-V-NEXT: csrr a0, vlenb
; CHECK-V-NEXT: slli a0, a0, 1
@@ -2691,19 +2693,19 @@ define <2 x i64> @utest_f32i64(<2 x float> %x) {
; CHECK-NOV-NEXT: .cfi_offset s0, -16
; CHECK-NOV-NEXT: .cfi_offset s1, -24
; CHECK-NOV-NEXT: .cfi_offset fs0, -32
-; CHECK-NOV-NEXT: fmv.s fs0, fa0
-; CHECK-NOV-NEXT: fmv.s fa0, fa1
+; CHECK-NOV-NEXT: fmv.s fs0, fa1
; CHECK-NOV-NEXT: call __fixunssfti
; CHECK-NOV-NEXT: mv s0, a0
; CHECK-NOV-NEXT: mv s1, a1
; CHECK-NOV-NEXT: fmv.s fa0, fs0
; CHECK-NOV-NEXT: call __fixunssfti
-; CHECK-NOV-NEXT: snez a2, s1
; CHECK-NOV-NEXT: snez a1, a1
+; CHECK-NOV-NEXT: snez a2, s1
+; CHECK-NOV-NEXT: addi a2, a2, -1
+; CHECK-NOV-NEXT: and a2, a2, s0
; CHECK-NOV-NEXT: addi a1, a1, -1
-; CHECK-NOV-NEXT: and a0, a1, a0
-; CHECK-NOV-NEXT: addi a1, a2, -1
-; CHECK-NOV-NEXT: and a1, a1, s0
+; CHECK-NOV-NEXT: and a1, a1, a0
+; CHECK-NOV-NEXT: mv a0, a2
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s1, 8(sp) # 8-byte Folded Reload
@@ -2728,25 +2730,25 @@ define <2 x i64> @utest_f32i64(<2 x float> %x) {
; CHECK-V-NEXT: addi a0, sp, 32
; CHECK-V-NEXT: vs1r.v v8, (a0) # Unknown-size Folded Spill
; CHECK-V-NEXT: vsetivli zero, 1, e32, mf2, ta, ma
-; CHECK-V-NEXT: vfmv.f.s fa0, v8
+; CHECK-V-NEXT: vslidedown.vi v9, v8, 1
+; CHECK-V-NEXT: vfmv.f.s fa0, v9
; CHECK-V-NEXT: call __fixunssfti
; CHECK-V-NEXT: mv s0, a0
; CHECK-V-NEXT: mv s1, a1
; CHECK-V-NEXT: vsetivli zero, 1, e32, mf2, ta, ma
; CHECK-V-NEXT: addi a0, sp, 32
; CHECK-V-NEXT: vl1r.v v8, (a0) # Unknown-size Folded Reload
-; CHECK-V-NEXT: vslidedown.vi v8, v8, 1
; CHECK-V-NEXT: vfmv.f.s fa0, v8
; CHECK-V-NEXT: call __fixunssfti
-; CHECK-V-NEXT: snez a2, s1
; CHECK-V-NEXT: snez a1, a1
-; CHECK-V-NEXT: addi a1, a1, -1
-; CHECK-V-NEXT: and a0, a1, a0
+; CHECK-V-NEXT: snez a2, s1
; CHECK-V-NEXT: addi a2, a2, -1
; CHECK-V-NEXT: and a2, a2, s0
+; CHECK-V-NEXT: addi a1, a1, -1
+; CHECK-V-NEXT: and a0, a1, a0
; CHECK-V-NEXT: vsetivli zero, 2, e64, m1, ta, ma
-; CHECK-V-NEXT: vmv.s.x v8, a2
-; CHECK-V-NEXT: vmv.s.x v9, a0
+; CHECK-V-NEXT: vmv.s.x v8, a0
+; CHECK-V-NEXT: vmv.s.x v9, a2
; CHECK-V-NEXT: vslideup.vi v8, v9, 1
; CHECK-V-NEXT: csrr a0, vlenb
; CHECK-V-NEXT: slli a0, a0, 1
@@ -2788,32 +2790,32 @@ define <2 x i64> @ustest_f32i64(<2 x float> %x) {
; CHECK-NOV-NEXT: # %bb.1: # %entry
; CHECK-NOV-NEXT: li a2, 1
; CHECK-NOV-NEXT: .LBB23_2: # %entry
-; CHECK-NOV-NEXT: slti a4, s1, 1
; CHECK-NOV-NEXT: slti a3, a1, 1
+; CHECK-NOV-NEXT: slti a4, s1, 1
; CHECK-NOV-NEXT: blez a1, .LBB23_4
; CHECK-NOV-NEXT: # %bb.3: # %entry
; CHECK-NOV-NEXT: li a1, 1
; CHECK-NOV-NEXT: .LBB23_4: # %entry
+; CHECK-NOV-NEXT: neg a4, a4
; CHECK-NOV-NEXT: neg a3, a3
; CHECK-NOV-NEXT: and a3, a3, a0
-; CHECK-NOV-NEXT: neg a0, a4
; CHECK-NOV-NEXT: beqz a1, .LBB23_7
; CHECK-NOV-NEXT: # %bb.5: # %entry
; CHECK-NOV-NEXT: sgtz a1, a1
-; CHECK-NOV-NEXT: and a0, a0, s0
+; CHECK-NOV-NEXT: and a4, a4, s0
; CHECK-NOV-NEXT: bnez a2, .LBB23_8
; CHECK-NOV-NEXT: .LBB23_6:
-; CHECK-NOV-NEXT: snez a2, a0
+; CHECK-NOV-NEXT: snez a0, a4
; CHECK-NOV-NEXT: j .LBB23_9
; CHECK-NOV-NEXT: .LBB23_7:
; CHECK-NOV-NEXT: snez a1, a3
-; CHECK-NOV-NEXT: and a0, a0, s0
+; CHECK-NOV-NEXT: and a4, a4, s0
; CHECK-NOV-NEXT: beqz a2, .LBB23_6
; CHECK-NOV-NEXT: .LBB23_8: # %entry
-; CHECK-NOV-NEXT: sgtz a2, a2
+; CHECK-NOV-NEXT: sgtz a0, a2
; CHECK-NOV-NEXT: .LBB23_9: # %entry
-; CHECK-NOV-NEXT: neg a2, a2
-; CHECK-NOV-NEXT: and a0, a2, a0
+; CHECK-NOV-NEXT: neg a0, a0
+; CHECK-NOV-NEXT: and a0, a0, a4
; CHECK-NOV-NEXT: neg a1, a1
; CHECK-NOV-NEXT: and a1, a1, a3
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
@@ -2855,15 +2857,15 @@ define <2 x i64> @ustest_f32i64(<2 x float> %x) {
; CHECK-V-NEXT: # %bb.1: # %entry
; CHECK-V-NEXT: li a2, 1
; CHECK-V-NEXT: .LBB23_2: # %entry
-; CHECK-V-NEXT: slti a3, s1, 1
; CHECK-V-NEXT: slti a4, a1, 1
+; CHECK-V-NEXT: slti a3, s1, 1
; CHECK-V-NEXT: blez a1, .LBB23_4
; CHECK-V-NEXT: # %bb.3: # %entry
; CHECK-V-NEXT: li a1, 1
; CHECK-V-NEXT: .LBB23_4: # %entry
+; CHECK-V-NEXT: neg a3, a3
; CHECK-V-NEXT: neg a4, a4
; CHECK-V-NEXT: and a0, a4, a0
-; CHECK-V-NEXT: neg a3, a3
; CHECK-V-NEXT: beqz a1, .LBB23_7
; CHECK-V-NEXT: # %bb.5: # %entry
; CHECK-V-NEXT: sgtz a1, a1
@@ -2918,8 +2920,8 @@ define <2 x i64> @stest_f16i64(<2 x half> %x) {
; CHECK-NOV-NEXT: .cfi_offset s0, -16
; CHECK-NOV-NEXT: .cfi_offset s1, -24
; CHECK-NOV-NEXT: .cfi_offset s2, -32
-; CHECK-NOV-NEXT: mv s2, a0
-; CHECK-NOV-NEXT: fmv.w.x fa0, a1
+; CHECK-NOV-NEXT: mv s2, a1
+; CHECK-NOV-NEXT: fmv.w.x fa0, a0
; CHECK-NOV-NEXT: call __extendhfsf2
; CHECK-NOV-NEXT: call __fixsfti
; CHECK-NOV-NEXT: mv s0, a0
@@ -2927,58 +2929,60 @@ define <2 x i64> @stest_f16i64(<2 x half> %x) {
; CHECK-NOV-NEXT: fmv.w.x fa0, s2
; CHECK-NOV-NEXT: call __extendhfsf2
; CHECK-NOV-NEXT: call __fixsfti
-; CHECK-NOV-NEXT: li a2, -1
-; CHECK-NOV-NEXT: srli a3, a2, 1
-; CHECK-NOV-NEXT: beqz s1, .LBB24_3
+; CHECK-NOV-NEXT: mv a2, a0
+; CHECK-NOV-NEXT: li a0, -1
+; CHECK-NOV-NEXT: srli a3, a0, 1
+; CHECK-NOV-NEXT: beqz a1, .LBB24_3
; CHECK-NOV-NEXT: # %bb.1: # %entry
-; CHECK-NOV-NEXT: slti a4, s1, 0
-; CHECK-NOV-NEXT: bnez a1, .LBB24_4
+; CHECK-NOV-NEXT: slti a4, a1, 0
+; CHECK-NOV-NEXT: bnez s1, .LBB24_4
; CHECK-NOV-NEXT: .LBB24_2:
-; CHECK-NOV-NEXT: sltu a5, a0, a3
+; CHECK-NOV-NEXT: sltu a5, s0, a3
; CHECK-NOV-NEXT: beqz a5, .LBB24_5
; CHECK-NOV-NEXT: j .LBB24_6
; CHECK-NOV-NEXT: .LBB24_3:
-; CHECK-NOV-NEXT: sltu a4, s0, a3
-; CHECK-NOV-NEXT: beqz a1, .LBB24_2
+; CHECK-NOV-NEXT: sltu a4, a2, a3
+; CHECK-NOV-NEXT: beqz s1, .LBB24_2
; CHECK-NOV-NEXT: .LBB24_4: # %entry
-; CHECK-NOV-NEXT: slti a5, a1, 0
+; CHECK-NOV-NEXT: slti a5, s1, 0
; CHECK-NOV-NEXT: bnez a5, .LBB24_6
; CHECK-NOV-NEXT: .LBB24_5: # %entry
-; CHECK-NOV-NEXT: mv a0, a3
+; CHECK-NOV-NEXT: mv s0, a3
; CHECK-NOV-NEXT: .LBB24_6: # %entry
; CHECK-NOV-NEXT: neg a6, a5
; CHECK-NOV-NEXT: neg a5, a4
-; CHECK-NOV-NEXT: and a5, a5, s1
+; CHECK-NOV-NEXT: and a5, a5, a1
; CHECK-NOV-NEXT: bnez a4, .LBB24_8
; CHECK-NOV-NEXT: # %bb.7: # %entry
-; CHECK-NOV-NEXT: mv s0, a3
+; CHECK-NOV-NEXT: mv a2, a3
; CHECK-NOV-NEXT: .LBB24_8: # %entry
-; CHECK-NOV-NEXT: and a4, a6, a1
-; CHECK-NOV-NEXT: slli a1, a2, 63
-; CHECK-NOV-NEXT: beq a5, a2, .LBB24_11
+; CHECK-NOV-NEXT: and a4, a6, s1
+; CHECK-NOV-NEXT: slli a1, a0, 63
+; CHECK-NOV-NEXT: beq a5, a0, .LBB24_11
; CHECK-NOV-NEXT: # %bb.9: # %entry
; CHECK-NOV-NEXT: slti a3, a5, 0
; CHECK-NOV-NEXT: xori a3, a3, 1
-; CHECK-NOV-NEXT: bne a4, a2, .LBB24_12
+; CHECK-NOV-NEXT: bne a4, a0, .LBB24_12
; CHECK-NOV-NEXT: .LBB24_10:
-; CHECK-NOV-NEXT: sltu a2, a1, a0
-; CHECK-NOV-NEXT: beqz a2, .LBB24_13
+; CHECK-NOV-NEXT: sltu a0, a1, s0
+; CHECK-NOV-NEXT: beqz a0, .LBB24_13
; CHECK-NOV-NEXT: j .LBB24_14
; CHECK-NOV-NEXT: .LBB24_11:
-; CHECK-NOV-NEXT: sltu a3, a1, s0
-; CHECK-NOV-NEXT: beq a4, a2, .LBB24_10
+; CHECK-NOV-NEXT: sltu a3, a1, a2
+; CHECK-NOV-NEXT: beq a4, a0, .LBB24_10
; CHECK-NOV-NEXT: .LBB24_12: # %entry
-; CHECK-NOV-NEXT: slti a2, a4, 0
-; CHECK-NOV-NEXT: xori a2, a2, 1
-; CHECK-NOV-NEXT: bnez a2, .LBB24_14
+; CHECK-NOV-NEXT: slti a0, a4, 0
+; CHECK-NOV-NEXT: xori a0, a0, 1
+; CHECK-NOV-NEXT: bnez a0, .LBB24_14
; CHECK-NOV-NEXT: .LBB24_13: # %entry
-; CHECK-NOV-NEXT: mv a0, a1
+; CHECK-NOV-NEXT: mv s0, a1
; CHECK-NOV-NEXT: .LBB24_14: # %entry
; CHECK-NOV-NEXT: bnez a3, .LBB24_16
; CHECK-NOV-NEXT: # %bb.15: # %entry
-; CHECK-NOV-NEXT: mv s0, a1
+; CHECK-NOV-NEXT: mv a2, a1
; CHECK-NOV-NEXT: .LBB24_16: # %entry
-; CHECK-NOV-NEXT: mv a1, s0
+; CHECK-NOV-NEXT: mv a0, s0
+; CHECK-NOV-NEXT: mv a1, a2
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s1, 8(sp) # 8-byte Folded Reload
@@ -2998,8 +3002,8 @@ define <2 x i64> @stest_f16i64(<2 x half> %x) {
; CHECK-V-NEXT: .cfi_offset s0, -16
; CHECK-V-NEXT: .cfi_offset s1, -24
; CHECK-V-NEXT: .cfi_offset s2, -32
-; CHECK-V-NEXT: mv s2, a0
-; CHECK-V-NEXT: fmv.w.x fa0, a1
+; CHECK-V-NEXT: mv s2, a1
+; CHECK-V-NEXT: fmv.w.x fa0, a0
; CHECK-V-NEXT: call __extendhfsf2
; CHECK-V-NEXT: call __fixsfti
; CHECK-V-NEXT: mv s0, a0
@@ -3009,31 +3013,31 @@ define <2 x i64> @stest_f16i64(<2 x half> %x) {
; CHECK-V-NEXT: call __fixsfti
; CHECK-V-NEXT: li a2, -1
; CHECK-V-NEXT: srli a3, a2, 1
-; CHECK-V-NEXT: beqz s1, .LBB24_3
+; CHECK-V-NEXT: beqz a1, .LBB24_3
; CHECK-V-NEXT: # %bb.1: # %entry
-; CHECK-V-NEXT: slti a4, s1, 0
-; CHECK-V-NEXT: bnez a1, .LBB24_4
+; CHECK-V-NEXT: slti a4, a1, 0
+; CHECK-V-NEXT: bnez s1, .LBB24_4
; CHECK-V-NEXT: .LBB24_2:
-; CHECK-V-NEXT: sltu a5, a0, a3
+; CHECK-V-NEXT: sltu a5, s0, a3
; CHECK-V-NEXT: beqz a5, .LBB24_5
; CHECK-V-NEXT: j .LBB24_6
; CHECK-V-NEXT: .LBB24_3:
-; CHECK-V-NEXT: sltu a4, s0, a3
-; CHECK-V-NEXT: beqz a1, .LBB24_2
+; CHECK-V-NEXT: sltu a4, a0, a3
+; CHECK-V-NEXT: beqz s1, .LBB24_2
; CHECK-V-NEXT: .LBB24_4: # %entry
-; CHECK-V-NEXT: slti a5, a1, 0
+; CHECK-V-NEXT: slti a5, s1, 0
; CHECK-V-NEXT: bnez a5, .LBB24_6
; CHECK-V-NEXT: .LBB24_5: # %entry
-; CHECK-V-NEXT: mv a0, a3
+; CHECK-V-NEXT: mv s0, a3
; CHECK-V-NEXT: .LBB24_6: # %entry
; CHECK-V-NEXT: neg a6, a5
; CHECK-V-NEXT: neg a5, a4
-; CHECK-V-NEXT: and a5, a5, s1
+; CHECK-V-NEXT: and a5, a5, a1
; CHECK-V-NEXT: bnez a4, .LBB24_8
; CHECK-V-NEXT: # %bb.7: # %entry
-; CHECK-V-NEXT: mv s0, a3
+; CHECK-V-NEXT: mv a0, a3
; CHECK-V-NEXT: .LBB24_8: # %entry
-; CHECK-V-NEXT: and a4, a6, a1
+; CHECK-V-NEXT: and a4, a6, s1
; CHECK-V-NEXT: slli a1, a2, 63
; CHECK-V-NEXT: beq a5, a2, .LBB24_11
; CHECK-V-NEXT: # %bb.9: # %entry
@@ -3041,26 +3045,26 @@ define <2 x i64> @stest_f16i64(<2 x half> %x) {
; CHECK-V-NEXT: xori a3, a3, 1
; CHECK-V-NEXT: bne a4, a2, .LBB24_12
; CHECK-V-NEXT: .LBB24_10:
-; CHECK-V-NEXT: sltu a2, a1, a0
+; CHECK-V-NEXT: sltu a2, a1, s0
; CHECK-V-NEXT: beqz a2, .LBB24_13
; CHECK-V-NEXT: j .LBB24_14
; CHECK-V-NEXT: .LBB24_11:
-; CHECK-V-NEXT: sltu a3, a1, s0
+; CHECK-V-NEXT: sltu a3, a1, a0
; CHECK-V-NEXT: beq a4, a2, .LBB24_10
; CHECK-V-NEXT: .LBB24_12: # %entry
; CHECK-V-NEXT: slti a2, a4, 0
; CHECK-V-NEXT: xori a2, a2, 1
; CHECK-V-NEXT: bnez a2, .LBB24_14
; CHECK-V-NEXT: .LBB24_13: # %entry
-; CHECK-V-NEXT: mv a0, a1
+; CHECK-V-NEXT: mv s0, a1
; CHECK-V-NEXT: .LBB24_14: # %entry
; CHECK-V-NEXT: bnez a3, .LBB24_16
; CHECK-V-NEXT: # %bb.15: # %entry
-; CHECK-V-NEXT: mv s0, a1
+; CHECK-V-NEXT: mv a0, a1
; CHECK-V-NEXT: .LBB24_16: # %entry
; CHECK-V-NEXT: vsetivli zero, 2, e64, m1, ta, ma
-; CHECK-V-NEXT: vmv.s.x v9, s0
-; CHECK-V-NEXT: vmv.s.x v8, a0
+; CHECK-V-NEXT: vmv.s.x v9, a0
+; CHECK-V-NEXT: vmv.s.x v8, s0
; CHECK-V-NEXT: vslideup.vi v8, v9, 1
; CHECK-V-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-V-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
@@ -3091,8 +3095,8 @@ define <2 x i64> @utesth_f16i64(<2 x half> %x) {
; CHECK-NOV-NEXT: .cfi_offset s0, -16
; CHECK-NOV-NEXT: .cfi_offset s1, -24
; CHECK-NOV-NEXT: .cfi_offset s2, -32
-; CHECK-NOV-NEXT: mv s0, a0
-; CHECK-NOV-NEXT: fmv.w.x fa0, a1
+; CHECK-NOV-NEXT: mv s0, a1
+; CHECK-NOV-NEXT: fmv.w.x fa0, a0
; CHECK-NOV-NEXT: call __extendhfsf2
; CHECK-NOV-NEXT: call __fixunssfti
; CHECK-NOV-NEXT: mv s1, a0
@@ -3100,12 +3104,13 @@ define <2 x i64> @utesth_f16i64(<2 x half> %x) {
; CHECK-NOV-NEXT: fmv.w.x fa0, s0
; CHECK-NOV-NEXT: call __extendhfsf2
; CHECK-NOV-NEXT: call __fixunssfti
-; CHECK-NOV-NEXT: snez a2, s2
; CHECK-NOV-NEXT: snez a1, a1
+; CHECK-NOV-NEXT: snez a2, s2
+; CHECK-NOV-NEXT: addi a2, a2, -1
+; CHECK-NOV-NEXT: and a2, a2, s1
; CHECK-NOV-NEXT: addi a1, a1, -1
-; CHECK-NOV-NEXT: and a0, a1, a0
-; CHECK-NOV-NEXT: addi a1, a2, -1
-; CHECK-NOV-NEXT: and a1, a1, s1
+; CHECK-NOV-NEXT: and a1, a1, a0
+; CHECK-NOV-NEXT: mv a0, a2
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s1, 8(sp) # 8-byte Folded Reload
@@ -3125,8 +3130,8 @@ define <2 x i64> @utesth_f16i64(<2 x half> %x) {
; CHECK-V-NEXT: .cfi_offset s0, -16
; CHECK-V-NEXT: .cfi_offset s1, -24
; CHECK-V-NEXT: .cfi_offset s2, -32
-; CHECK-V-NEXT: mv s0, a0
-; CHECK-V-NEXT: fmv.w.x fa0, a1
+; CHECK-V-NEXT: mv s0, a1
+; CHECK-V-NEXT: fmv.w.x fa0, a0
; CHECK-V-NEXT: call __extendhfsf2
; CHECK-V-NEXT: call __fixunssfti
; CHECK-V-NEXT: mv s1, a0
@@ -3134,15 +3139,15 @@ define <2 x i64> @utesth_f16i64(<2 x half> %x) {
; CHECK-V-NEXT: fmv.w.x fa0, s0
; CHECK-V-NEXT: call __extendhfsf2
; CHECK-V-NEXT: call __fixunssfti
-; CHECK-V-NEXT: snez a2, s2
; CHECK-V-NEXT: snez a1, a1
-; CHECK-V-NEXT: addi a1, a1, -1
-; CHECK-V-NEXT: and a0, a1, a0
+; CHECK-V-NEXT: snez a2, s2
; CHECK-V-NEXT: addi a2, a2, -1
; CHECK-V-NEXT: and a2, a2, s1
+; CHECK-V-NEXT: addi a1, a1, -1
+; CHECK-V-NEXT: and a0, a1, a0
; CHECK-V-NEXT: vsetivli zero, 2, e64, m1, ta, ma
-; CHECK-V-NEXT: vmv.s.x v9, a2
-; CHECK-V-NEXT: vmv.s.x v8, a0
+; CHECK-V-NEXT: vmv.s.x v9, a0
+; CHECK-V-NEXT: vmv.s.x v8, a2
; CHECK-V-NEXT: vslideup.vi v8, v9, 1
; CHECK-V-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-V-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
@@ -3185,32 +3190,32 @@ define <2 x i64> @ustest_f16i64(<2 x half> %x) {
; CHECK-NOV-NEXT: # %bb.1: # %entry
; CHECK-NOV-NEXT: li a2, 1
; CHECK-NOV-NEXT: .LBB26_2: # %entry
-; CHECK-NOV-NEXT: slti a4, s1, 1
; CHECK-NOV-NEXT: slti a3, a1, 1
+; CHECK-NOV-NEXT: slti a4, s1, 1
; CHECK-NOV-NEXT: blez a1, .LBB26_4
; CHECK-NOV-NEXT: # %bb.3: # %entry
; CHECK-NOV-NEXT: li a1, 1
; CHECK-NOV-NEXT: .LBB26_4: # %entry
+; CHECK-NOV-NEXT: neg a4, a4
; CHECK-NOV-NEXT: neg a3, a3
; CHECK-NOV-NEXT: and a3, a3, a0
-; CHECK-NOV-NEXT: neg a0, a4
; CHECK-NOV-NEXT: beqz a1, .LBB26_7
; CHECK-NOV-NEXT: # %bb.5: # %entry
; CHECK-NOV-NEXT: sgtz a1, a1
-; CHECK-NOV-NEXT: and a0, a0, s0
+; CHECK-NOV-NEXT: and a4, a4, s0
; CHECK-NOV-NEXT: bnez a2, .LBB26_8
; CHECK-NOV-NEXT: .LBB26_6:
-; CHECK-NOV-NEXT: snez a2, a0
+; CHECK-NOV-NEXT: snez a0, a4
; CHECK-NOV-NEXT: j .LBB26_9
; CHECK-NOV-NEXT: .LBB26_7:
; CHECK-NOV-NEXT: snez a1, a3
-; CHECK-NOV-NEXT: and a0, a0, s0
+; CHECK-NOV-NEXT: and a4, a4, s0
; CHECK-NOV-NEXT: beqz a2, .LBB26_6
; CHECK-NOV-NEXT: .LBB26_8: # %entry
-; CHECK-NOV-NEXT: sgtz a2, a2
+; CHECK-NOV-NEXT: sgtz a0, a2
; CHECK-NOV-NEXT: .LBB26_9: # %entry
-; CHECK-NOV-NEXT: neg a2, a2
-; CHECK-NOV-NEXT: and a0, a2, a0
+; CHECK-NOV-NEXT: neg a0, a0
+; CHECK-NOV-NEXT: and a0, a0, a4
; CHECK-NOV-NEXT: neg a1, a1
; CHECK-NOV-NEXT: and a1, a1, a3
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
@@ -3246,15 +3251,15 @@ define <2 x i64> @ustest_f16i64(<2 x half> %x) {
; CHECK-V-NEXT: # %bb.1: # %entry
; CHECK-V-NEXT: li a2, 1
; CHECK-V-NEXT: .LBB26_2: # %entry
-; CHECK-V-NEXT: slti a3, s1, 1
; CHECK-V-NEXT: slti a4, a1, 1
+; CHECK-V-NEXT: slti a3, s1, 1
; CHECK-V-NEXT: blez a1, .LBB26_4
; CHECK-V-NEXT: # %bb.3: # %entry
; CHECK-V-NEXT: li a1, 1
; CHECK-V-NEXT: .LBB26_4: # %entry
+; CHECK-V-NEXT: neg a3, a3
; CHECK-V-NEXT: neg a4, a4
; CHECK-V-NEXT: and a0, a4, a0
-; CHECK-V-NEXT: neg a3, a3
; CHECK-V-NEXT: beqz a1, .LBB26_7
; CHECK-V-NEXT: # %bb.5: # %entry
; CHECK-V-NEXT: sgtz a1, a1
@@ -5643,15 +5648,15 @@ define <2 x i64> @ustest_f64i64_mm(<2 x double> %x) {
; CHECK-NOV-NEXT: mv s1, a1
; CHECK-NOV-NEXT: fmv.d fa0, fs0
; CHECK-NOV-NEXT: call __fixdfti
-; CHECK-NOV-NEXT: mv a2, s1
-; CHECK-NOV-NEXT: mv a3, a1
+; CHECK-NOV-NEXT: mv a2, a1
; CHECK-NOV-NEXT: blez a1, .LBB47_2
; CHECK-NOV-NEXT: # %bb.1: # %entry
-; CHECK-NOV-NEXT: li a3, 1
+; CHECK-NOV-NEXT: li a2, 1
; CHECK-NOV-NEXT: .LBB47_2: # %entry
-; CHECK-NOV-NEXT: blez a2, .LBB47_4
+; CHECK-NOV-NEXT: mv a3, s1
+; CHECK-NOV-NEXT: blez s1, .LBB47_4
; CHECK-NOV-NEXT: # %bb.3: # %entry
-; CHECK-NOV-NEXT: li a2, 1
+; CHECK-NOV-NEXT: li a3, 1
; CHECK-NOV-NEXT: .LBB47_4: # %entry
; CHECK-NOV-NEXT: slti a1, a1, 1
; CHECK-NOV-NEXT: neg a1, a1
@@ -5659,11 +5664,11 @@ define <2 x i64> @ustest_f64i64_mm(<2 x double> %x) {
; CHECK-NOV-NEXT: slti a0, s1, 1
; CHECK-NOV-NEXT: neg a0, a0
; CHECK-NOV-NEXT: and a0, a0, s0
+; CHECK-NOV-NEXT: slti a3, a3, 0
+; CHECK-NOV-NEXT: addi a3, a3, -1
+; CHECK-NOV-NEXT: and a0, a3, a0
; CHECK-NOV-NEXT: slti a2, a2, 0
; CHECK-NOV-NEXT: addi a2, a2, -1
-; CHECK-NOV-NEXT: and a0, a2, a0
-; CHECK-NOV-NEXT: slti a2, a3, 0
-; CHECK-NOV-NEXT: addi a2, a2, -1
; CHECK-NOV-NEXT: and a1, a2, a1
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
@@ -5699,15 +5704,15 @@ define <2 x i64> @ustest_f64i64_mm(<2 x double> %x) {
; CHECK-V-NEXT: vl1r.v v8, (a0) # Unknown-size Folded Reload
; CHECK-V-NEXT: vfmv.f.s fa0, v8
; CHECK-V-NEXT: call __fixdfti
-; CHECK-V-NEXT: mv a2, s1
-; CHECK-V-NEXT: mv a3, a1
+; CHECK-V-NEXT: mv a2, a1
; CHECK-V-NEXT: blez a1, .LBB47_2
; CHECK-V-NEXT: # %bb.1: # %entry
-; CHECK-V-NEXT: li a3, 1
+; CHECK-V-NEXT: li a2, 1
; CHECK-V-NEXT: .LBB47_2: # %entry
-; CHECK-V-NEXT: blez a2, .LBB47_4
+; CHECK-V-NEXT: mv a3, s1
+; CHECK-V-NEXT: blez s1, .LBB47_4
; CHECK-V-NEXT: # %bb.3: # %entry
-; CHECK-V-NEXT: li a2, 1
+; CHECK-V-NEXT: li a3, 1
; CHECK-V-NEXT: .LBB47_4: # %entry
; CHECK-V-NEXT: slti a1, a1, 1
; CHECK-V-NEXT: neg a1, a1
@@ -5715,11 +5720,11 @@ define <2 x i64> @ustest_f64i64_mm(<2 x double> %x) {
; CHECK-V-NEXT: slti a1, s1, 1
; CHECK-V-NEXT: neg a1, a1
; CHECK-V-NEXT: and a1, a1, s0
+; CHECK-V-NEXT: slti a3, a3, 0
+; CHECK-V-NEXT: addi a3, a3, -1
+; CHECK-V-NEXT: and a1, a3, a1
; CHECK-V-NEXT: slti a2, a2, 0
; CHECK-V-NEXT: addi a2, a2, -1
-; CHECK-V-NEXT: and a1, a2, a1
-; CHECK-V-NEXT: slti a2, a3, 0
-; CHECK-V-NEXT: addi a2, a2, -1
; CHECK-V-NEXT: and a0, a2, a0
; CHECK-V-NEXT: vsetivli zero, 2, e64, m1, ta, ma
; CHECK-V-NEXT: vmv.s.x v8, a0
@@ -6029,15 +6034,15 @@ define <2 x i64> @ustest_f32i64_mm(<2 x float> %x) {
; CHECK-NOV-NEXT: mv s1, a1
; CHECK-NOV-NEXT: fmv.s fa0, fs0
; CHECK-NOV-NEXT: call __fixsfti
-; CHECK-NOV-NEXT: mv a2, s1
-; CHECK-NOV-NEXT: mv a3, a1
+; CHECK-NOV-NEXT: mv a2, a1
; CHECK-NOV-NEXT: blez a1, .LBB50_2
; CHECK-NOV-NEXT: # %bb.1: # %entry
-; CHECK-NOV-NEXT: li a3, 1
+; CHECK-NOV-NEXT: li a2, 1
; CHECK-NOV-NEXT: .LBB50_2: # %entry
-; CHECK-NOV-NEXT: blez a2, .LBB50_4
+; CHECK-NOV-NEXT: mv a3, s1
+; CHECK-NOV-NEXT: blez s1, .LBB50_4
; CHECK-NOV-NEXT: # %bb.3: # %entry
-; CHECK-NOV-NEXT: li a2, 1
+; CHECK-NOV-NEXT: li a3, 1
; CHECK-NOV-NEXT: .LBB50_4: # %entry
; CHECK-NOV-NEXT: slti a1, a1, 1
; CHECK-NOV-NEXT: neg a1, a1
@@ -6045,11 +6050,11 @@ define <2 x i64> @ustest_f32i64_mm(<2 x float> %x) {
; CHECK-NOV-NEXT: slti a0, s1, 1
; CHECK-NOV-NEXT: neg a0, a0
; CHECK-NOV-NEXT: and a0, a0, s0
+; CHECK-NOV-NEXT: slti a3, a3, 0
+; CHECK-NOV-NEXT: addi a3, a3, -1
+; CHECK-NOV-NEXT: and a0, a3, a0
; CHECK-NOV-NEXT: slti a2, a2, 0
; CHECK-NOV-NEXT: addi a2, a2, -1
-; CHECK-NOV-NEXT: and a0, a2, a0
-; CHECK-NOV-NEXT: slti a2, a3, 0
-; CHECK-NOV-NEXT: addi a2, a2, -1
; CHECK-NOV-NEXT: and a1, a2, a1
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
@@ -6085,15 +6090,15 @@ define <2 x i64> @ustest_f32i64_mm(<2 x float> %x) {
; CHECK-V-NEXT: vl1r.v v8, (a0) # Unknown-size Folded Reload
; CHECK-V-NEXT: vfmv.f.s fa0, v8
; CHECK-V-NEXT: call __fixsfti
-; CHECK-V-NEXT: mv a2, s1
-; CHECK-V-NEXT: mv a3, a1
+; CHECK-V-NEXT: mv a2, a1
; CHECK-V-NEXT: blez a1, .LBB50_2
; CHECK-V-NEXT: # %bb.1: # %entry
-; CHECK-V-NEXT: li a3, 1
+; CHECK-V-NEXT: li a2, 1
; CHECK-V-NEXT: .LBB50_2: # %entry
-; CHECK-V-NEXT: blez a2, .LBB50_4
+; CHECK-V-NEXT: mv a3, s1
+; CHECK-V-NEXT: blez s1, .LBB50_4
; CHECK-V-NEXT: # %bb.3: # %entry
-; CHECK-V-NEXT: li a2, 1
+; CHECK-V-NEXT: li a3, 1
; CHECK-V-NEXT: .LBB50_4: # %entry
; CHECK-V-NEXT: slti a1, a1, 1
; CHECK-V-NEXT: neg a1, a1
@@ -6101,11 +6106,11 @@ define <2 x i64> @ustest_f32i64_mm(<2 x float> %x) {
; CHECK-V-NEXT: slti a1, s1, 1
; CHECK-V-NEXT: neg a1, a1
; CHECK-V-NEXT: and a1, a1, s0
+; CHECK-V-NEXT: slti a3, a3, 0
+; CHECK-V-NEXT: addi a3, a3, -1
+; CHECK-V-NEXT: and a1, a3, a1
; CHECK-V-NEXT: slti a2, a2, 0
; CHECK-V-NEXT: addi a2, a2, -1
-; CHECK-V-NEXT: and a1, a2, a1
-; CHECK-V-NEXT: slti a2, a3, 0
-; CHECK-V-NEXT: addi a2, a2, -1
; CHECK-V-NEXT: and a0, a2, a0
; CHECK-V-NEXT: vsetivli zero, 2, e64, m1, ta, ma
; CHECK-V-NEXT: vmv.s.x v8, a0
@@ -6407,15 +6412,15 @@ define <2 x i64> @ustest_f16i64_mm(<2 x half> %x) {
; CHECK-NOV-NEXT: fmv.w.x fa0, s2
; CHECK-NOV-NEXT: call __extendhfsf2
; CHECK-NOV-NEXT: call __fixsfti
-; CHECK-NOV-NEXT: mv a2, s1
-; CHECK-NOV-NEXT: mv a3, a1
+; CHECK-NOV-NEXT: mv a2, a1
; CHECK-NOV-NEXT: blez a1, .LBB53_2
; CHECK-NOV-NEXT: # %bb.1: # %entry
-; CHECK-NOV-NEXT: li a3, 1
+; CHECK-NOV-NEXT: li a2, 1
; CHECK-NOV-NEXT: .LBB53_2: # %entry
-; CHECK-NOV-NEXT: blez a2, .LBB53_4
+; CHECK-NOV-NEXT: mv a3, s1
+; CHECK-NOV-NEXT: blez s1, .LBB53_4
; CHECK-NOV-NEXT: # %bb.3: # %entry
-; CHECK-NOV-NEXT: li a2, 1
+; CHECK-NOV-NEXT: li a3, 1
; CHECK-NOV-NEXT: .LBB53_4: # %entry
; CHECK-NOV-NEXT: slti a1, a1, 1
; CHECK-NOV-NEXT: neg a1, a1
@@ -6423,11 +6428,11 @@ define <2 x i64> @ustest_f16i64_mm(<2 x half> %x) {
; CHECK-NOV-NEXT: slti a0, s1, 1
; CHECK-NOV-NEXT: neg a0, a0
; CHECK-NOV-NEXT: and a0, a0, s0
+; CHECK-NOV-NEXT: slti a3, a3, 0
+; CHECK-NOV-NEXT: addi a3, a3, -1
+; CHECK-NOV-NEXT: and a0, a3, a0
; CHECK-NOV-NEXT: slti a2, a2, 0
; CHECK-NOV-NEXT: addi a2, a2, -1
-; CHECK-NOV-NEXT: and a0, a2, a0
-; CHECK-NOV-NEXT: slti a2, a3, 0
-; CHECK-NOV-NEXT: addi a2, a2, -1
; CHECK-NOV-NEXT: and a1, a2, a1
; CHECK-NOV-NEXT: ld ra, 24(sp) # 8-byte Folded Reload
; CHECK-NOV-NEXT: ld s0, 16(sp) # 8-byte Folded Reload
@@ -6457,15 +6462,15 @@ define <2 x i64> @ustest_f16i64_mm(<2 x half> %x) {
; CHECK-V-NEXT: fmv.w.x fa0, s2
; CHECK-V-NEXT: call __extendhfsf2
; CHECK-V-NEXT: call __fixsfti
-; CHECK-V-NEXT: mv a2, s1
-; CHECK-V-NEXT: mv a3, a1
+; CHECK-V-NEXT: mv a2, a1
; CHECK-V-NEXT: blez a1, .LBB53_2
; CHECK-V-NEXT: # %bb.1: # %entry
-; CHECK-V-NEXT: li a3, 1
+; CHECK-V-NEXT: li a2, 1
; CHECK-V-NEXT: .LBB53_2: # %entry
-; CHECK-V-NEXT: blez a2, .LBB53_4
+; CHECK-V-NEXT: mv a3, s1
+; CHECK-V-NEXT: blez s1, .LBB53_4
; CHECK-V-NEXT: # %bb.3: # %entry
-; CHECK-V-NEXT: li a2, 1
+; CHECK-V-NEXT: li a3, 1
; CHECK-V-NEXT: .LBB53_4: # %entry
; CHECK-V-NEXT: slti a1, a1, 1
; CHECK-V-NEXT: neg a1, a1
@@ -6473,11 +6478,11 @@ define <2 x i64> @ustest_f16i64_mm(<2 x half> %x) {
; CHECK-V-NEXT: slti a1, s1, 1
; CHECK-V-NEXT: neg a1, a1
; CHECK-V-NEXT: and a1, a1, s0
+; CHECK-V-NEXT: slti a3, a3, 0
+; CHECK-V-NEXT: addi a3, a3, -1
+; CHECK-V-NEXT: and a1, a3, a1
; CHECK-V-NEXT: slti a2, a2, 0
; CHECK-V-NEXT: addi a2, a2, -1
-; CHECK-V-NEXT: and a1, a2, a1
-; CHECK-V-NEXT: slti a2, a3, 0
-; CHECK-V-NEXT: addi a2, a2, -1
; CHECK-V-NEXT: and a0, a2, a0
; CHECK-V-NEXT: vsetivli zero, 2, e64, m1, ta, ma
; CHECK-V-NEXT: vmv.s.x v9, a0
diff --git a/llvm/test/CodeGen/X86/abdu-vector-128.ll b/llvm/test/CodeGen/X86/abdu-vector-128.ll
index dd180b67e492a0..0c33e8973c2d20 100644
--- a/llvm/test/CodeGen/X86/abdu-vector-128.ll
+++ b/llvm/test/CodeGen/X86/abdu-vector-128.ll
@@ -715,43 +715,41 @@ define <2 x i64> @abd_cmp_v2i64_multiuse_cmp(<2 x i64> %a, <2 x i64> %b) nounwin
;
; SSE42-LABEL: abd_cmp_v2i64_multiuse_cmp:
; SSE42: # %bb.0:
-; SSE42-NEXT: movdqa {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808]
+; SSE42-NEXT: movdqa %xmm0, %xmm2
+; SSE42-NEXT: psubq %xmm1, %xmm2
; SSE42-NEXT: movdqa %xmm1, %xmm3
-; SSE42-NEXT: pxor %xmm2, %xmm3
-; SSE42-NEXT: pxor %xmm0, %xmm2
-; SSE42-NEXT: pcmpgtq %xmm3, %xmm2
-; SSE42-NEXT: movdqa %xmm0, %xmm3
-; SSE42-NEXT: psubq %xmm1, %xmm3
-; SSE42-NEXT: psubq %xmm0, %xmm1
-; SSE42-NEXT: movdqa %xmm2, %xmm0
-; SSE42-NEXT: blendvpd %xmm0, %xmm3, %xmm1
-; SSE42-NEXT: paddq %xmm1, %xmm2
-; SSE42-NEXT: movdqa %xmm2, %xmm0
+; SSE42-NEXT: psubq %xmm0, %xmm3
+; SSE42-NEXT: movdqa {{.*#+}} xmm4 = [9223372036854775808,9223372036854775808]
+; SSE42-NEXT: pxor %xmm4, %xmm1
+; SSE42-NEXT: pxor %xmm4, %xmm0
+; SSE42-NEXT: pcmpgtq %xmm1, %xmm0
+; SSE42-NEXT: blendvpd %xmm0, %xmm2, %xmm3
+; SSE42-NEXT: paddq %xmm3, %xmm0
; SSE42-NEXT: retq
;
; AVX1-LABEL: abd_cmp_v2i64_multiuse_cmp:
; AVX1: # %bb.0:
-; AVX1-NEXT: vmovddup {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808]
-; AVX1-NEXT: # xmm2 = mem[0,0]
-; AVX1-NEXT: vpxor %xmm2, %xmm1, %xmm3
-; AVX1-NEXT: vpxor %xmm2, %xmm0, %xmm2
-; AVX1-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2
-; AVX1-NEXT: vpsubq %xmm1, %xmm0, %xmm3
-; AVX1-NEXT: vpsubq %xmm0, %xmm1, %xmm0
-; AVX1-NEXT: vblendvpd %xmm2, %xmm3, %xmm0, %xmm0
-; AVX1-NEXT: vpaddq %xmm0, %xmm2, %xmm0
+; AVX1-NEXT: vpsubq %xmm1, %xmm0, %xmm2
+; AVX1-NEXT: vpsubq %xmm0, %xmm1, %xmm3
+; AVX1-NEXT: vmovddup {{.*#+}} xmm4 = [9223372036854775808,9223372036854775808]
+; AVX1-NEXT: # xmm4 = mem[0,0]
+; AVX1-NEXT: vpxor %xmm4, %xmm1, %xmm1
+; AVX1-NEXT: vpxor %xmm4, %xmm0, %xmm0
+; AVX1-NEXT: vpcmpgtq %xmm1, %xmm0, %xmm0
+; AVX1-NEXT: vblendvpd %xmm0, %xmm2, %xmm3, %xmm1
+; AVX1-NEXT: vpaddq %xmm1, %xmm0, %xmm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: abd_cmp_v2i64_multiuse_cmp:
; AVX2: # %bb.0:
-; AVX2-NEXT: vpbroadcastq {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808]
-; AVX2-NEXT: vpxor %xmm2, %xmm1, %xmm3
-; AVX2-NEXT: vpxor %xmm2, %xmm0, %xmm2
-; AVX2-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2
-; AVX2-NEXT: vpsubq %xmm1, %xmm0, %xmm3
-; AVX2-NEXT: vpsubq %xmm0, %xmm1, %xmm0
-; AVX2-NEXT: vblendvpd %xmm2, %xmm3, %xmm0, %xmm0
-; AVX2-NEXT: vpaddq %xmm0, %xmm2, %xmm0
+; AVX2-NEXT: vpsubq %xmm1, %xmm0, %xmm2
+; AVX2-NEXT: vpsubq %xmm0, %xmm1, %xmm3
+; AVX2-NEXT: vpbroadcastq {{.*#+}} xmm4 = [9223372036854775808,9223372036854775808]
+; AVX2-NEXT: vpxor %xmm4, %xmm1, %xmm1
+; AVX2-NEXT: vpxor %xmm4, %xmm0, %xmm0
+; AVX2-NEXT: vpcmpgtq %xmm1, %xmm0, %xmm0
+; AVX2-NEXT: vblendvpd %xmm0, %xmm2, %xmm3, %xmm1
+; AVX2-NEXT: vpaddq %xmm1, %xmm0, %xmm0
; AVX2-NEXT: retq
;
; AVX512-LABEL: abd_cmp_v2i64_multiuse_cmp:
diff --git a/llvm/test/CodeGen/X86/apx/kmov-postrapseudos.ll b/llvm/test/CodeGen/X86/apx/kmov-postrapseudos.ll
index 017024c173c3f7..b2cb2c3e04b3f4 100644
--- a/llvm/test/CodeGen/X86/apx/kmov-postrapseudos.ll
+++ b/llvm/test/CodeGen/X86/apx/kmov-postrapseudos.ll
@@ -52,10 +52,7 @@ alloca_21:
define i32 @kmovrk_1(<4 x ptr> %arg) {
; AVX512-LABEL: kmovrk_1:
; AVX512: # %bb.0: # %bb
-; AVX512-NEXT: # kill: def $ymm0 killed $ymm0 def $zmm0
-; AVX512-NEXT: vptestmq %zmm0, %zmm0, %k0 # encoding: [0x62,0xf2,0xfd,0x48,0x27,0xc0]
-; AVX512-NEXT: kmovw %k0, %eax # EVEX TO VEX Compression encoding: [0xc5,0xf8,0x93,0xc0]
-; AVX512-NEXT: testb $15, %al # encoding: [0xa8,0x0f]
+; AVX512-NEXT: vptest %ymm0, %ymm0 # encoding: [0xc4,0xe2,0x7d,0x17,0xc0]
; AVX512-NEXT: jne .LBB2_1 # encoding: [0x75,A]
; AVX512-NEXT: # fixup A - offset: 1, value: .LBB2_1-1, kind: FK_PCRel_1
; AVX512-NEXT: # %bb.2: # %bb3
@@ -66,10 +63,7 @@ define i32 @kmovrk_1(<4 x ptr> %arg) {
;
; AVX512BW-LABEL: kmovrk_1:
; AVX512BW: # %bb.0: # %bb
-; AVX512BW-NEXT: # kill: def $ymm0 killed $ymm0 def $zmm0
-; AVX512BW-NEXT: vptestmq %zmm0, %zmm0, %k0 # encoding: [0x62,0xf2,0xfd,0x48,0x27,0xc0]
-; AVX512BW-NEXT: kmovd %k0, %eax # EVEX TO VEX Compression encoding: [0xc5,0xfb,0x93,0xc0]
-; AVX512BW-NEXT: testb $15, %al # encoding: [0xa8,0x0f]
+; AVX512BW-NEXT: vptest %ymm0, %ymm0 # encoding: [0xc4,0xe2,0x7d,0x17,0xc0]
; AVX512BW-NEXT: jne .LBB2_1 # encoding: [0x75,A]
; AVX512BW-NEXT: # fixup A - offset: 1, value: .LBB2_1-1, kind: FK_PCRel_1
; AVX512BW-NEXT: # %bb.2: # %bb3
diff --git a/llvm/test/CodeGen/X86/avx512-broadcast-arith.ll b/llvm/test/CodeGen/X86/avx512-broadcast-arith.ll
index 13d1265a249d1f..7e48b3719cf0ff 100644
--- a/llvm/test/CodeGen/X86/avx512-broadcast-arith.ll
+++ b/llvm/test/CodeGen/X86/avx512-broadcast-arith.ll
@@ -30,13 +30,13 @@ define <64 x i8> @add_v64i8_broadcasts(<64 x i8> %a0, i64 %a1, i8 %a2) {
; AVX512F-NEXT: vinserti128 $1, %xmm4, %ymm3, %ymm3
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm3, %zmm2
; AVX512F-NEXT: vpternlogq $216, %zmm2, %zmm1, %zmm0
-; AVX512F-NEXT: vpaddb %ymm1, %ymm0, %ymm3
-; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm0
+; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm3
+; AVX512F-NEXT: vpaddb %ymm1, %ymm3, %ymm3
; AVX512F-NEXT: vpaddb %ymm1, %ymm0, %ymm0
-; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm3, %zmm4
+; AVX512F-NEXT: vinserti64x4 $1, %ymm3, %zmm0, %zmm4
+; AVX512F-NEXT: vpaddb %ymm1, %ymm3, %ymm3
; AVX512F-NEXT: vpaddb %ymm1, %ymm0, %ymm0
-; AVX512F-NEXT: vpaddb %ymm1, %ymm3, %ymm1
-; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0
+; AVX512F-NEXT: vinserti64x4 $1, %ymm3, %zmm0, %zmm0
; AVX512F-NEXT: vpternlogq $226, %zmm4, %zmm2, %zmm0
; AVX512F-NEXT: retq
;
diff --git a/llvm/test/CodeGen/X86/combine-mul.ll b/llvm/test/CodeGen/X86/combine-mul.ll
index 8d2bb77a9e1af6..5d7bf4a2c9788f 100644
--- a/llvm/test/CodeGen/X86/combine-mul.ll
+++ b/llvm/test/CodeGen/X86/combine-mul.ll
@@ -80,13 +80,13 @@ define <4 x i32> @combine_vec_mul_pow2b(<4 x i32> %x) {
define <4 x i64> @combine_vec_mul_pow2c(<4 x i64> %x) {
; SSE-LABEL: combine_vec_mul_pow2c:
; SSE: # %bb.0:
+; SSE-NEXT: movdqa %xmm0, %xmm2
+; SSE-NEXT: paddq %xmm0, %xmm2
+; SSE-NEXT: pblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm2[4,5,6,7]
; SSE-NEXT: movdqa %xmm1, %xmm2
; SSE-NEXT: psllq $4, %xmm2
; SSE-NEXT: psllq $2, %xmm1
; SSE-NEXT: pblendw {{.*#+}} xmm1 = xmm1[0,1,2,3],xmm2[4,5,6,7]
-; SSE-NEXT: movdqa %xmm0, %xmm2
-; SSE-NEXT: paddq %xmm0, %xmm2
-; SSE-NEXT: pblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm2[4,5,6,7]
; SSE-NEXT: retq
;
; AVX-LABEL: combine_vec_mul_pow2c:
@@ -399,14 +399,12 @@ define i64 @combine_mul_self_demandedbits(i64 %x) {
; SSE: # %bb.0:
; SSE-NEXT: movq %rdi, %rax
; SSE-NEXT: imulq %rdi, %rax
-; SSE-NEXT: andq $-3, %rax
; SSE-NEXT: retq
;
; AVX-LABEL: combine_mul_self_demandedbits:
; AVX: # %bb.0:
; AVX-NEXT: movq %rdi, %rax
; AVX-NEXT: imulq %rdi, %rax
-; AVX-NEXT: andq $-3, %rax
; AVX-NEXT: retq
%1 = mul i64 %x, %x
%2 = and i64 %1, -3
diff --git a/llvm/test/CodeGen/X86/div-rem-pair-recomposition-signed.ll b/llvm/test/CodeGen/X86/div-rem-pair-recomposition-signed.ll
index e12ca56023a7f2..bf7c1c00c71df1 100644
--- a/llvm/test/CodeGen/X86/div-rem-pair-recomposition-signed.ll
+++ b/llvm/test/CodeGen/X86/div-rem-pair-recomposition-signed.ll
@@ -178,15 +178,15 @@ define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
; X86-NEXT: pushl %edi
; X86-NEXT: pushl %esi
; X86-NEXT: subl $152, %esp
-; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ebp
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movl %ecx, %eax
; X86-NEXT: sarl $31, %eax
-; X86-NEXT: movl %edx, %edi
-; X86-NEXT: sarl $31, %edi
+; X86-NEXT: movl %ebp, %edx
+; X86-NEXT: sarl $31, %edx
; X86-NEXT: movl %eax, %esi
; X86-NEXT: xorl %ecx, %esi
-; X86-NEXT: movl %esi, %ebp
+; X86-NEXT: movl %esi, %edi
; X86-NEXT: movl %eax, %ecx
; X86-NEXT: xorl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movl %ecx, %ebx
@@ -195,67 +195,66 @@ define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
; X86-NEXT: movl %eax, %esi
; X86-NEXT: xorl {{[0-9]+}}(%esp), %esi
; X86-NEXT: subl %eax, %esi
-; X86-NEXT: movl %esi, (%esp) # 4-byte Spill
+; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: sbbl %eax, %ecx
; X86-NEXT: movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: sbbl %eax, %ebx
-; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: sbbl %eax, %ebp
-; X86-NEXT: movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %edi, %esi
-; X86-NEXT: xorl %edx, %esi
-; X86-NEXT: movl %edi, %edx
-; X86-NEXT: xorl {{[0-9]+}}(%esp), %edx
-; X86-NEXT: movl %edi, %ebx
+; X86-NEXT: movl %ebx, (%esp) # 4-byte Spill
+; X86-NEXT: sbbl %eax, %edi
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl %edx, %edi
+; X86-NEXT: xorl %ebp, %edi
+; X86-NEXT: movl %edx, %ebx
; X86-NEXT: xorl {{[0-9]+}}(%esp), %ebx
-; X86-NEXT: movl %edi, %ebp
+; X86-NEXT: movl %edx, %esi
+; X86-NEXT: xorl {{[0-9]+}}(%esp), %esi
+; X86-NEXT: movl %edx, %ebp
; X86-NEXT: xorl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT: subl %edi, %ebp
-; X86-NEXT: sbbl %edi, %ebx
-; X86-NEXT: sbbl %edi, %edx
-; X86-NEXT: sbbl %edi, %esi
-; X86-NEXT: xorl %eax, %edi
-; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %ebx, %eax
-; X86-NEXT: orl %esi, %eax
+; X86-NEXT: subl %edx, %ebp
+; X86-NEXT: sbbl %edx, %esi
+; X86-NEXT: sbbl %edx, %ebx
+; X86-NEXT: sbbl %edx, %edi
+; X86-NEXT: xorl %eax, %edx
+; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl %esi, %eax
+; X86-NEXT: orl %edi, %eax
; X86-NEXT: movl %ebp, %ecx
-; X86-NEXT: orl %edx, %ecx
-; X86-NEXT: movl %edx, %edi
+; X86-NEXT: orl %ebx, %ecx
; X86-NEXT: orl %eax, %ecx
; X86-NEXT: sete %cl
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; X86-NEXT: movl (%esp), %edx # 4-byte Reload
-; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT: orl (%esp), %edx # 4-byte Folded Reload
; X86-NEXT: orl %eax, %edx
; X86-NEXT: sete %al
; X86-NEXT: orb %cl, %al
; X86-NEXT: movb %al, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
-; X86-NEXT: bsrl %esi, %edx
+; X86-NEXT: bsrl %edi, %edx
; X86-NEXT: xorl $31, %edx
-; X86-NEXT: bsrl %edi, %ecx
+; X86-NEXT: bsrl %ebx, %ecx
; X86-NEXT: xorl $31, %ecx
; X86-NEXT: addl $32, %ecx
-; X86-NEXT: testl %esi, %esi
+; X86-NEXT: testl %edi, %edi
; X86-NEXT: cmovnel %edx, %ecx
-; X86-NEXT: bsrl %ebx, %edx
+; X86-NEXT: bsrl %esi, %edx
; X86-NEXT: xorl $31, %edx
; X86-NEXT: movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: bsrl %ebp, %ebp
; X86-NEXT: xorl $31, %ebp
; X86-NEXT: addl $32, %ebp
-; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: testl %ebx, %ebx
+; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: testl %esi, %esi
; X86-NEXT: cmovnel %edx, %ebp
; X86-NEXT: addl $64, %ebp
+; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: orl %esi, %edi
+; X86-NEXT: orl %edi, %ebx
; X86-NEXT: cmovnel %ecx, %ebp
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
; X86-NEXT: bsrl %edi, %edx
; X86-NEXT: xorl $31, %edx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT: movl (%esp), %eax # 4-byte Reload
; X86-NEXT: bsrl %eax, %ecx
; X86-NEXT: xorl $31, %ecx
; X86-NEXT: addl $32, %ecx
@@ -264,7 +263,7 @@ define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
; X86-NEXT: bsrl %ebx, %esi
; X86-NEXT: xorl $31, %esi
-; X86-NEXT: bsrl (%esp), %edx # 4-byte Folded Reload
+; X86-NEXT: bsrl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
; X86-NEXT: xorl $31, %edx
; X86-NEXT: addl $32, %edx
; X86-NEXT: testl %ebx, %ebx
@@ -272,52 +271,56 @@ define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
; X86-NEXT: addl $64, %edx
; X86-NEXT: movl %eax, %esi
; X86-NEXT: orl %edi, %esi
+; X86-NEXT: movl %edi, %ebx
; X86-NEXT: cmovnel %ecx, %edx
; X86-NEXT: xorl %esi, %esi
; X86-NEXT: subl %edx, %ebp
-; X86-NEXT: movl $0, %ebx
-; X86-NEXT: sbbl %ebx, %ebx
; X86-NEXT: movl $0, %edx
; X86-NEXT: sbbl %edx, %edx
; X86-NEXT: movl $0, %eax
; X86-NEXT: sbbl %eax, %eax
+; X86-NEXT: movl $0, %edi
+; X86-NEXT: sbbl %edi, %edi
; X86-NEXT: movl $127, %ecx
; X86-NEXT: movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: cmpl %ebp, %ecx
; X86-NEXT: movl $0, %ecx
-; X86-NEXT: sbbl %ebx, %ecx
-; X86-NEXT: movl $0, %ecx
; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: sbbl %edx, %ecx
; X86-NEXT: movl $0, %ecx
; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: sbbl %eax, %ecx
+; X86-NEXT: movl $0, %ecx
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: sbbl %edi, %ecx
; X86-NEXT: setb %cl
; X86-NEXT: orb {{[-0-9]+}}(%e{{[sb]}}p), %cl # 1-byte Folded Reload
+; X86-NEXT: movl %ebx, %edi
; X86-NEXT: cmovnel %esi, %edi
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT: movl (%esp), %edx # 4-byte Reload
; X86-NEXT: cmovnel %esi, %edx
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: cmovnel %esi, %eax
-; X86-NEXT: cmovel (%esp), %esi # 4-byte Folded Reload
+; X86-NEXT: cmovel {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
; X86-NEXT: movl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT: jne .LBB4_8
-; X86-NEXT: # %bb.1: # %_udiv-special-cases
-; X86-NEXT: movl %ebx, %ecx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT: xorl $127, %ebx
-; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
-; X86-NEXT: movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: jne .LBB4_1
+; X86-NEXT: # %bb.8: # %_udiv-special-cases
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
+; X86-NEXT: xorl $127, %ebp
+; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Folded Reload
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; X86-NEXT: orl %ebx, %ecx
-; X86-NEXT: je .LBB4_8
-; X86-NEXT: # %bb.2: # %udiv-bb1
-; X86-NEXT: movl (%esp), %eax # 4-byte Reload
-; X86-NEXT: movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT: orl %ebp, %ecx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ebp
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: je .LBB4_9
+; X86-NEXT: # %bb.5: # %udiv-bb1
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: movl %eax, {{[0-9]+}}(%esp)
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT: movl (%esp), %eax # 4-byte Reload
+; X86-NEXT: movl %eax, {{[0-9]+}}(%esp)
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: movl %eax, {{[0-9]+}}(%esp)
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
@@ -332,78 +335,82 @@ define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
; X86-NEXT: shrb $3, %al
; X86-NEXT: andb $15, %al
; X86-NEXT: negb %al
-; X86-NEXT: movsbl %al, %ebx
-; X86-NEXT: movl 144(%esp,%ebx), %edx
-; X86-NEXT: movl 148(%esp,%ebx), %edi
+; X86-NEXT: movsbl %al, %edi
+; X86-NEXT: movl 144(%esp,%edi), %edx
+; X86-NEXT: movl 148(%esp,%edi), %esi
; X86-NEXT: movb %ch, %cl
-; X86-NEXT: shldl %cl, %edx, %edi
+; X86-NEXT: shldl %cl, %edx, %esi
+; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: shll %cl, %edx
; X86-NEXT: notb %cl
-; X86-NEXT: movl 140(%esp,%ebx), %eax
+; X86-NEXT: movl 140(%esp,%edi), %eax
; X86-NEXT: movl %eax, %esi
; X86-NEXT: shrl %esi
; X86-NEXT: shrl %cl, %esi
; X86-NEXT: orl %edx, %esi
; X86-NEXT: movl %esi, %edx
-; X86-NEXT: movl 136(%esp,%ebx), %esi
+; X86-NEXT: movl 136(%esp,%edi), %esi
; X86-NEXT: movb %ch, %cl
; X86-NEXT: shldl %cl, %esi, %eax
; X86-NEXT: shll %cl, %esi
-; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: addl $1, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
; X86-NEXT: adcl $0, %ecx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT: adcl $0, %ebx
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
+; X86-NEXT: adcl $0, %edi
; X86-NEXT: adcl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT: jae .LBB4_3
+; X86-NEXT: jae .LBB4_2
; X86-NEXT: # %bb.6:
-; X86-NEXT: xorl %ebx, %ebx
-; X86-NEXT: xorl %esi, %esi
+; X86-NEXT: movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; X86-NEXT: xorl %ecx, %ecx
; X86-NEXT: jmp .LBB4_7
-; X86-NEXT: .LBB4_3: # %udiv-preheader
-; X86-NEXT: movl (%esp), %esi # 4-byte Reload
-; X86-NEXT: movl %esi, {{[0-9]+}}(%esp)
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT: movl %esi, {{[0-9]+}}(%esp)
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT: movl %esi, {{[0-9]+}}(%esp)
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT: movl %esi, {{[0-9]+}}(%esp)
+; X86-NEXT: .LBB4_1:
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: jmp .LBB4_9
+; X86-NEXT: .LBB4_2: # %udiv-preheader
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: movl %ebx, {{[0-9]+}}(%esp)
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: movl %ebx, {{[0-9]+}}(%esp)
+; X86-NEXT: movl (%esp), %ebx # 4-byte Reload
+; X86-NEXT: movl %ebx, {{[0-9]+}}(%esp)
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: movl %ebx, {{[0-9]+}}(%esp)
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
-; X86-NEXT: movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movb %bl, %ch
; X86-NEXT: andb $7, %ch
; X86-NEXT: movb %bl, %cl
; X86-NEXT: shrb $3, %cl
; X86-NEXT: andb $15, %cl
-; X86-NEXT: movzbl %cl, %ebp
-; X86-NEXT: movl 100(%esp,%ebp), %esi
-; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl 96(%esp,%ebp), %ebx
-; X86-NEXT: movl %ebp, %eax
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movzbl %cl, %ebx
+; X86-NEXT: movl 100(%esp,%ebx), %ebp
+; X86-NEXT: movl %ebp, (%esp) # 4-byte Spill
+; X86-NEXT: movl 96(%esp,%ebx), %edi
; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %ebx, %edx
+; X86-NEXT: movl %edi, %edx
; X86-NEXT: movb %ch, %cl
-; X86-NEXT: shrdl %cl, %esi, %edx
-; X86-NEXT: movl 88(%esp,%ebp), %ebp
-; X86-NEXT: movl 92(%esp,%eax), %esi
-; X86-NEXT: movl %esi, %eax
+; X86-NEXT: shrdl %cl, %ebp, %edx
+; X86-NEXT: movl %edx, %ebp
+; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl 88(%esp,%ebx), %edx
+; X86-NEXT: movl 92(%esp,%ebx), %ebx
+; X86-NEXT: movl %ebx, %eax
; X86-NEXT: shrl %cl, %eax
; X86-NEXT: notb %cl
-; X86-NEXT: addl %ebx, %ebx
-; X86-NEXT: shll %cl, %ebx
-; X86-NEXT: orl %eax, %ebx
-; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: addl %edi, %edi
+; X86-NEXT: shll %cl, %edi
+; X86-NEXT: orl %eax, %edi
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movb %ch, %cl
-; X86-NEXT: shrl %cl, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT: shrdl %cl, %esi, %ebp
+; X86-NEXT: shrl %cl, (%esp) # 4-byte Folded Spill
+; X86-NEXT: shrdl %cl, %ebx, %edx
+; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: addl $-1, %eax
; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
@@ -413,113 +420,109 @@ define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: adcl $-1, %eax
; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT: movl %ecx, %eax
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: adcl $-1, %eax
; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: xorl %esi, %esi
; X86-NEXT: movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
; X86-NEXT: .p2align 4, 0x90
-; X86-NEXT: .LBB4_4: # %udiv-do-while
+; X86-NEXT: .LBB4_3: # %udiv-do-while
; X86-NEXT: # =>This Inner Loop Header: Depth=1
-; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT: shldl $1, %edx, %ebx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT: shldl $1, %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; X86-NEXT: movl %ebp, %edx
+; X86-NEXT: shldl $1, %ebp, (%esp) # 4-byte Folded Spill
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
; X86-NEXT: shldl $1, %ebp, %edx
-; X86-NEXT: shldl $1, %edi, %ebp
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT: shldl $1, %eax, %edi
-; X86-NEXT: orl %esi, %edi
-; X86-NEXT: movl %edi, (%esp) # 4-byte Spill
-; X86-NEXT: movl %ecx, %edi
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: shldl $1, %ebx, %ebp
+; X86-NEXT: shldl $1, %edi, %ebx
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT: shldl $1, %ecx, %eax
-; X86-NEXT: orl %esi, %eax
-; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: shldl $1, %ecx, %edi
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT: shldl $1, %eax, %ecx
-; X86-NEXT: orl %esi, %ecx
+; X86-NEXT: orl %eax, %edi
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
+; X86-NEXT: shldl $1, %edi, %ecx
+; X86-NEXT: orl %eax, %ecx
; X86-NEXT: movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: addl %eax, %eax
-; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: cmpl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
+; X86-NEXT: shldl $1, %esi, %edi
+; X86-NEXT: orl %eax, %edi
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: addl %esi, %esi
+; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
+; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: cmpl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT: sbbl %edx, %ecx
+; X86-NEXT: sbbl %ebp, %ecx
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT: sbbl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
+; X86-NEXT: sbbl %edx, %ecx
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT: sbbl %ebx, %ecx
+; X86-NEXT: sbbl (%esp), %ecx # 4-byte Folded Reload
; X86-NEXT: sarl $31, %ecx
-; X86-NEXT: movl %ecx, %eax
-; X86-NEXT: andl $1, %eax
-; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl %ecx, %esi
-; X86-NEXT: andl %edi, %esi
+; X86-NEXT: andl $1, %esi
+; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl %ecx, %esi
+; X86-NEXT: andl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
; X86-NEXT: movl %ecx, %edi
; X86-NEXT: andl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
; X86-NEXT: movl %ecx, %eax
; X86-NEXT: andl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
; X86-NEXT: andl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; X86-NEXT: subl %ecx, %ebp
-; X86-NEXT: sbbl %eax, %edx
-; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT: sbbl %edi, %edx
-; X86-NEXT: movl (%esp), %edi # 4-byte Reload
-; X86-NEXT: sbbl %esi, %ebx
+; X86-NEXT: subl %ecx, %ebx
; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: sbbl %eax, %ebp
+; X86-NEXT: movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: sbbl %edi, %edx
+; X86-NEXT: movl %edx, %ebp
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT: sbbl %esi, (%esp) # 4-byte Folded Spill
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
; X86-NEXT: addl $-1, %ecx
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: adcl $-1, %eax
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT: adcl $-1, %esi
-; X86-NEXT: adcl $-1, %ebx
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
+; X86-NEXT: adcl $-1, %edi
+; X86-NEXT: adcl $-1, %edx
; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: orl %ebx, %eax
+; X86-NEXT: orl %edx, %eax
; X86-NEXT: movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: orl %esi, %ecx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: orl %edi, %ecx
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
; X86-NEXT: orl %eax, %ecx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT: jne .LBB4_4
-; X86-NEXT: # %bb.5:
+; X86-NEXT: jne .LBB4_3
+; X86-NEXT: # %bb.4:
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: movl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
; X86-NEXT: .LBB4_7: # %udiv-loop-exit
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
; X86-NEXT: shldl $1, %edx, %edi
-; X86-NEXT: orl %esi, %edi
+; X86-NEXT: orl %ecx, %edi
; X86-NEXT: shldl $1, %eax, %edx
-; X86-NEXT: orl %esi, %edx
-; X86-NEXT: movl %esi, %ecx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT: orl %ecx, %edx
; X86-NEXT: shldl $1, %esi, %eax
; X86-NEXT: orl %ecx, %eax
; X86-NEXT: addl %esi, %esi
-; X86-NEXT: orl %ebx, %esi
-; X86-NEXT: .LBB4_8: # %udiv-end
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT: xorl %ecx, %edi
-; X86-NEXT: xorl %ecx, %edx
-; X86-NEXT: xorl %ecx, %eax
-; X86-NEXT: xorl %ecx, %esi
-; X86-NEXT: subl %ecx, %esi
+; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
+; X86-NEXT: .LBB4_9: # %udiv-end
+; X86-NEXT: xorl %ebx, %edi
+; X86-NEXT: xorl %ebx, %edx
+; X86-NEXT: xorl %ebx, %eax
+; X86-NEXT: xorl %ebx, %esi
+; X86-NEXT: subl %ebx, %esi
; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: sbbl %ecx, %eax
+; X86-NEXT: sbbl %ebx, %eax
; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: sbbl %ecx, %edx
-; X86-NEXT: sbbl %ecx, %edi
-; X86-NEXT: movl %edi, (%esp) # 4-byte Spill
+; X86-NEXT: sbbl %ebx, %edx
+; X86-NEXT: sbbl %ebx, %edi
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl %esi, (%ebp)
; X86-NEXT: movl %eax, 4(%ebp)
; X86-NEXT: movl %edx, 8(%ebp)
@@ -532,7 +535,7 @@ define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl %esi, %eax
; X86-NEXT: mull %edi
-; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl %eax, (%esp) # 4-byte Spill
; X86-NEXT: movl %edx, %edi
; X86-NEXT: addl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
; X86-NEXT: adcl $0, %ecx
@@ -553,10 +556,10 @@ define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
; X86-NEXT: adcl %eax, %edx
; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl (%esp), %ecx # 4-byte Reload
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
; X86-NEXT: imull %eax, %ecx
; X86-NEXT: mull %ebx
-; X86-NEXT: movl %eax, (%esp) # 4-byte Spill
+; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: imull {{[0-9]+}}(%esp), %ebx
; X86-NEXT: addl %edx, %ebx
; X86-NEXT: addl %ecx, %ebx
@@ -568,12 +571,12 @@ define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
; X86-NEXT: mull %edx
; X86-NEXT: addl %edx, %ebp
; X86-NEXT: addl %ecx, %ebp
-; X86-NEXT: addl (%esp), %eax # 4-byte Folded Reload
+; X86-NEXT: addl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
; X86-NEXT: adcl %ebx, %ebp
; X86-NEXT: addl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
; X86-NEXT: adcl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Folded Reload
; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT: subl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
+; X86-NEXT: subl (%esp), %edx # 4-byte Folded Reload
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: sbbl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
diff --git a/llvm/test/CodeGen/X86/fold-masked-merge.ll b/llvm/test/CodeGen/X86/fold-masked-merge.ll
index 135494ac25f8cb..b2614c5fe0493c 100644
--- a/llvm/test/CodeGen/X86/fold-masked-merge.ll
+++ b/llvm/test/CodeGen/X86/fold-masked-merge.ll
@@ -56,9 +56,7 @@ define i8 @masked_merge2(i8 %a0, i8 %a1, i8 %a2) {
; NOBMI-LABEL: masked_merge2:
; NOBMI: # %bb.0:
; NOBMI-NEXT: movl %esi, %eax
-; NOBMI-NEXT: xorb %sil, %al
-; NOBMI-NEXT: andb %dil, %al
-; NOBMI-NEXT: xorb %sil, %al
+; NOBMI-NEXT: # kill: def $al killed $al killed $eax
; NOBMI-NEXT: retq
;
; BMI-LABEL: masked_merge2:
diff --git a/llvm/test/CodeGen/X86/freeze-binary.ll b/llvm/test/CodeGen/X86/freeze-binary.ll
index b212e9438e1b52..d75fc5318bd8aa 100644
--- a/llvm/test/CodeGen/X86/freeze-binary.ll
+++ b/llvm/test/CodeGen/X86/freeze-binary.ll
@@ -202,13 +202,27 @@ define <4 x i32> @freeze_add_vec(<4 x i32> %a0) nounwind {
define <4 x i32> @freeze_add_vec_undef(<4 x i32> %a0) nounwind {
; X86-LABEL: freeze_add_vec_undef:
; X86: # %bb.0:
+; X86-NEXT: pushl %ebp
+; X86-NEXT: movl %esp, %ebp
+; X86-NEXT: andl $-16, %esp
+; X86-NEXT: subl $32, %esp
+; X86-NEXT: movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT: movl $3, {{[0-9]+}}(%esp)
+; X86-NEXT: movl $2, {{[0-9]+}}(%esp)
+; X86-NEXT: movl $1, (%esp)
+; X86-NEXT: paddd (%esp), %xmm0
; X86-NEXT: paddd {{\.?LCPI[0-9]+_[0-9]+}}, %xmm0
-; X86-NEXT: paddd {{\.?LCPI[0-9]+_[0-9]+}}, %xmm0
+; X86-NEXT: movl %ebp, %esp
+; X86-NEXT: popl %ebp
; X86-NEXT: retl
;
; X64-LABEL: freeze_add_vec_undef:
; X64: # %bb.0:
-; X64-NEXT: vpaddd {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
+; X64-NEXT: movabsq $8589934593, %rax # imm = 0x200000001
+; X64-NEXT: movq %rax, -{{[0-9]+}}(%rsp)
+; X64-NEXT: movl %eax, -{{[0-9]+}}(%rsp)
+; X64-NEXT: movl $3, -{{[0-9]+}}(%rsp)
+; X64-NEXT: vpaddd -{{[0-9]+}}(%rsp), %xmm0, %xmm0
; X64-NEXT: vpaddd {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
; X64-NEXT: retq
%x = add <4 x i32> %a0, <i32 1, i32 2, i32 3, i32 undef>
@@ -273,13 +287,27 @@ define <4 x i32> @freeze_sub_vec(<4 x i32> %a0) nounwind {
define <4 x i32> @freeze_sub_vec_undef(<4 x i32> %a0) nounwind {
; X86-LABEL: freeze_sub_vec_undef:
; X86: # %bb.0:
+; X86-NEXT: pushl %ebp
+; X86-NEXT: movl %esp, %ebp
+; X86-NEXT: andl $-16, %esp
+; X86-NEXT: subl $32, %esp
+; X86-NEXT: movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT: movl $3, {{[0-9]+}}(%esp)
+; X86-NEXT: movl $2, {{[0-9]+}}(%esp)
+; X86-NEXT: movl $1, (%esp)
+; X86-NEXT: psubd (%esp), %xmm0
; X86-NEXT: psubd {{\.?LCPI[0-9]+_[0-9]+}}, %xmm0
-; X86-NEXT: psubd {{\.?LCPI[0-9]+_[0-9]+}}, %xmm0
+; X86-NEXT: movl %ebp, %esp
+; X86-NEXT: popl %ebp
; X86-NEXT: retl
;
; X64-LABEL: freeze_sub_vec_undef:
; X64: # %bb.0:
-; X64-NEXT: vpsubd {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
+; X64-NEXT: movabsq $8589934593, %rax # imm = 0x200000001
+; X64-NEXT: movq %rax, -{{[0-9]+}}(%rsp)
+; X64-NEXT: movl %eax, -{{[0-9]+}}(%rsp)
+; X64-NEXT: movl $3, -{{[0-9]+}}(%rsp)
+; X64-NEXT: vpsubd -{{[0-9]+}}(%rsp), %xmm0, %xmm0
; X64-NEXT: vpsubd {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
; X64-NEXT: retq
%x = sub <4 x i32> %a0, <i32 1, i32 2, i32 3, i32 undef>
@@ -345,13 +373,29 @@ define <8 x i16> @freeze_mul_vec(<8 x i16> %a0) nounwind {
define <8 x i16> @freeze_mul_vec_undef(<8 x i16> %a0) nounwind {
; X86-LABEL: freeze_mul_vec_undef:
; X86: # %bb.0:
+; X86-NEXT: pushl %ebp
+; X86-NEXT: movl %esp, %ebp
+; X86-NEXT: andl $-16, %esp
+; X86-NEXT: subl $32, %esp
+; X86-NEXT: movw %ax, {{[0-9]+}}(%esp)
+; X86-NEXT: movw $1, {{[0-9]+}}(%esp)
+; X86-NEXT: movl $196612, {{[0-9]+}}(%esp) # imm = 0x30004
+; X86-NEXT: movl $262147, {{[0-9]+}}(%esp) # imm = 0x40003
+; X86-NEXT: movl $131073, (%esp) # imm = 0x20001
+; X86-NEXT: pmullw (%esp), %xmm0
; X86-NEXT: pmullw {{\.?LCPI[0-9]+_[0-9]+}}, %xmm0
-; X86-NEXT: pmullw {{\.?LCPI[0-9]+_[0-9]+}}, %xmm0
+; X86-NEXT: movl %ebp, %esp
+; X86-NEXT: popl %ebp
; X86-NEXT: retl
;
; X64-LABEL: freeze_mul_vec_undef:
; X64: # %bb.0:
-; X64-NEXT: vpmullw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
+; X64-NEXT: movabsq $1125912791875585, %rax # imm = 0x4000300020001
+; X64-NEXT: movq %rax, -{{[0-9]+}}(%rsp)
+; X64-NEXT: movw %ax, -{{[0-9]+}}(%rsp)
+; X64-NEXT: movw $1, -{{[0-9]+}}(%rsp)
+; X64-NEXT: movl $196612, -{{[0-9]+}}(%rsp) # imm = 0x30004
+; X64-NEXT: vpmullw -{{[0-9]+}}(%rsp), %xmm0, %xmm0
; X64-NEXT: vpmullw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
; X64-NEXT: retq
%x = mul <8 x i16> %a0, <i16 1, i16 2, i16 3, i16 4, i16 4, i16 3, i16 undef, i16 1>
diff --git a/llvm/test/CodeGen/X86/freeze-combine.ll b/llvm/test/CodeGen/X86/freeze-combine.ll
index b037a6d9a1b93b..1cfb8627a4dd45 100644
--- a/llvm/test/CodeGen/X86/freeze-combine.ll
+++ b/llvm/test/CodeGen/X86/freeze-combine.ll
@@ -3,9 +3,9 @@
define i32 @const() {
; CHECK-LABEL: name: const
; CHECK: bb.0 (%ir-block.0):
- ; CHECK: [[MOV32ri:%[0-9]+]]:gr32 = MOV32ri 1
- ; CHECK: $eax = COPY [[MOV32ri]]
- ; CHECK: RET 0, $eax
+ ; CHECK-NEXT: [[MOV32ri:%[0-9]+]]:gr32 = MOV32ri 1
+ ; CHECK-NEXT: $eax = COPY [[MOV32ri]]
+ ; CHECK-NEXT: RET 0, $eax
%y = freeze i32 1
ret i32 %y
}
@@ -13,11 +13,11 @@ define i32 @const() {
define i32 @fold(i32 %x) {
; CHECK-LABEL: name: fold
; CHECK: bb.0 (%ir-block.0):
- ; CHECK: liveins: $edi
- ; CHECK: [[COPY:%[0-9]+]]:gr32 = COPY $edi
- ; CHECK: [[COPY1:%[0-9]+]]:gr32 = COPY [[COPY]]
- ; CHECK: $eax = COPY [[COPY1]]
- ; CHECK: RET 0, $eax
+ ; CHECK-NEXT: liveins: $edi
+ ; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:gr32 = COPY $edi
+ ; CHECK-NEXT: $eax = COPY [[COPY]]
+ ; CHECK-NEXT: RET 0, $eax
%y = freeze i32 %x
%z = freeze i32 %y
ret i32 %z
diff --git a/llvm/test/CodeGen/X86/gfni-funnel-shifts.ll b/llvm/test/CodeGen/X86/gfni-funnel-shifts.ll
index 8f1272013c11b6..451d25930fbba3 100644
--- a/llvm/test/CodeGen/X86/gfni-funnel-shifts.ll
+++ b/llvm/test/CodeGen/X86/gfni-funnel-shifts.ll
@@ -41,17 +41,17 @@ declare <16 x i8> @llvm.fshl.v16i8(<16 x i8>, <16 x i8>, <16 x i8>)
define <16 x i8> @splatconstant_fshr_v16i8(<16 x i8> %a, <16 x i8> %b) nounwind {
; GFNISSE-LABEL: splatconstant_fshr_v16i8:
; GFNISSE: # %bb.0:
+; GFNISSE-NEXT: paddb %xmm0, %xmm0
; GFNISSE-NEXT: psrlw $7, %xmm1
; GFNISSE-NEXT: pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm1
-; GFNISSE-NEXT: paddb %xmm0, %xmm0
; GFNISSE-NEXT: por %xmm1, %xmm0
; GFNISSE-NEXT: retq
;
; GFNIAVX1OR2-LABEL: splatconstant_fshr_v16i8:
; GFNIAVX1OR2: # %bb.0:
+; GFNIAVX1OR2-NEXT: vpaddb %xmm0, %xmm0, %xmm0
; GFNIAVX1OR2-NEXT: vpsrlw $7, %xmm1, %xmm1
; GFNIAVX1OR2-NEXT: vpand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm1, %xmm1
-; GFNIAVX1OR2-NEXT: vpaddb %xmm0, %xmm0, %xmm0
; GFNIAVX1OR2-NEXT: vpor %xmm1, %xmm0, %xmm0
; GFNIAVX1OR2-NEXT: retq
;
diff --git a/llvm/test/CodeGen/X86/gfni-rotates.ll b/llvm/test/CodeGen/X86/gfni-rotates.ll
index 8b94a3f84cb960..05c87be6678a76 100644
--- a/llvm/test/CodeGen/X86/gfni-rotates.ll
+++ b/llvm/test/CodeGen/X86/gfni-rotates.ll
@@ -43,18 +43,18 @@ define <16 x i8> @splatconstant_rotr_v16i8(<16 x i8> %a) nounwind {
; GFNISSE-LABEL: splatconstant_rotr_v16i8:
; GFNISSE: # %bb.0:
; GFNISSE-NEXT: movdqa %xmm0, %xmm1
-; GFNISSE-NEXT: psrlw $7, %xmm1
-; GFNISSE-NEXT: pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm1
-; GFNISSE-NEXT: paddb %xmm0, %xmm0
+; GFNISSE-NEXT: paddb %xmm0, %xmm1
+; GFNISSE-NEXT: psrlw $7, %xmm0
+; GFNISSE-NEXT: pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
; GFNISSE-NEXT: por %xmm1, %xmm0
; GFNISSE-NEXT: retq
;
; GFNIAVX1OR2-LABEL: splatconstant_rotr_v16i8:
; GFNIAVX1OR2: # %bb.0:
-; GFNIAVX1OR2-NEXT: vpsrlw $7, %xmm0, %xmm1
-; GFNIAVX1OR2-NEXT: vpand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm1, %xmm1
-; GFNIAVX1OR2-NEXT: vpaddb %xmm0, %xmm0, %xmm0
-; GFNIAVX1OR2-NEXT: vpor %xmm1, %xmm0, %xmm0
+; GFNIAVX1OR2-NEXT: vpaddb %xmm0, %xmm0, %xmm1
+; GFNIAVX1OR2-NEXT: vpsrlw $7, %xmm0, %xmm0
+; GFNIAVX1OR2-NEXT: vpand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
+; GFNIAVX1OR2-NEXT: vpor %xmm0, %xmm1, %xmm0
; GFNIAVX1OR2-NEXT: retq
;
; GFNIAVX512-LABEL: splatconstant_rotr_v16i8:
diff --git a/llvm/test/CodeGen/X86/known-never-zero.ll b/llvm/test/CodeGen/X86/known-never-zero.ll
index cc9862769f2b66..61edbaa8f16f87 100644
--- a/llvm/test/CodeGen/X86/known-never-zero.ll
+++ b/llvm/test/CodeGen/X86/known-never-zero.ll
@@ -325,12 +325,13 @@ define i32 @rotr_known_nonzero(i32 %xx, i32 %y) {
; CHECK: # %bb.0:
; CHECK-NEXT: movl %esi, %ecx
; CHECK-NEXT: orl $256, %edi # imm = 0x100
+; CHECK-NEXT: movl %edi, %eax
; CHECK-NEXT: # kill: def $cl killed $cl killed $ecx
-; CHECK-NEXT: rorl %cl, %edi
+; CHECK-NEXT: rorl %cl, %eax
; CHECK-NEXT: testl %edi, %edi
; CHECK-NEXT: je .LBB18_1
; CHECK-NEXT: # %bb.2: # %cond.false
-; CHECK-NEXT: rep bsfl %edi, %eax
+; CHECK-NEXT: rep bsfl %eax, %eax
; CHECK-NEXT: retq
; CHECK-NEXT: .LBB18_1:
; CHECK-NEXT: movl $32, %eax
@@ -348,12 +349,13 @@ define i32 @rotr_maybe_zero(i32 %x, i32 %y) {
; CHECK-LABEL: rotr_maybe_zero:
; CHECK: # %bb.0:
; CHECK-NEXT: movl %esi, %ecx
+; CHECK-NEXT: movl %edi, %eax
; CHECK-NEXT: # kill: def $cl killed $cl killed $ecx
-; CHECK-NEXT: rorl %cl, %edi
+; CHECK-NEXT: rorl %cl, %eax
; CHECK-NEXT: testl %edi, %edi
; CHECK-NEXT: je .LBB19_1
; CHECK-NEXT: # %bb.2: # %cond.false
-; CHECK-NEXT: rep bsfl %edi, %eax
+; CHECK-NEXT: rep bsfl %eax, %eax
; CHECK-NEXT: retq
; CHECK-NEXT: .LBB19_1:
; CHECK-NEXT: movl $32, %eax
@@ -385,12 +387,13 @@ define i32 @rotr_with_fshr_maybe_zero(i32 %x, i32 %y) {
; CHECK-LABEL: rotr_with_fshr_maybe_zero:
; CHECK: # %bb.0:
; CHECK-NEXT: movl %esi, %ecx
+; CHECK-NEXT: movl %edi, %eax
; CHECK-NEXT: # kill: def $cl killed $cl killed $ecx
-; CHECK-NEXT: rorl %cl, %edi
+; CHECK-NEXT: rorl %cl, %eax
; CHECK-NEXT: testl %edi, %edi
; CHECK-NEXT: je .LBB21_1
; CHECK-NEXT: # %bb.2: # %cond.false
-; CHECK-NEXT: rep bsfl %edi, %eax
+; CHECK-NEXT: rep bsfl %eax, %eax
; CHECK-NEXT: retq
; CHECK-NEXT: .LBB21_1:
; CHECK-NEXT: movl $32, %eax
@@ -405,12 +408,13 @@ define i32 @rotl_known_nonzero(i32 %xx, i32 %y) {
; CHECK: # %bb.0:
; CHECK-NEXT: movl %esi, %ecx
; CHECK-NEXT: orl $256, %edi # imm = 0x100
+; CHECK-NEXT: movl %edi, %eax
; CHECK-NEXT: # kill: def $cl killed $cl killed $ecx
-; CHECK-NEXT: roll %cl, %edi
+; CHECK-NEXT: roll %cl, %eax
; CHECK-NEXT: testl %edi, %edi
; CHECK-NEXT: je .LBB22_1
; CHECK-NEXT: # %bb.2: # %cond.false
-; CHECK-NEXT: rep bsfl %edi, %eax
+; CHECK-NEXT: rep bsfl %eax, %eax
; CHECK-NEXT: retq
; CHECK-NEXT: .LBB22_1:
; CHECK-NEXT: movl $32, %eax
@@ -428,12 +432,13 @@ define i32 @rotl_maybe_zero(i32 %x, i32 %y) {
; CHECK-LABEL: rotl_maybe_zero:
; CHECK: # %bb.0:
; CHECK-NEXT: movl %esi, %ecx
+; CHECK-NEXT: movl %edi, %eax
; CHECK-NEXT: # kill: def $cl killed $cl killed $ecx
-; CHECK-NEXT: roll %cl, %edi
+; CHECK-NEXT: roll %cl, %eax
; CHECK-NEXT: testl %edi, %edi
; CHECK-NEXT: je .LBB23_1
; CHECK-NEXT: # %bb.2: # %cond.false
-; CHECK-NEXT: rep bsfl %edi, %eax
+; CHECK-NEXT: rep bsfl %eax, %eax
; CHECK-NEXT: retq
; CHECK-NEXT: .LBB23_1:
; CHECK-NEXT: movl $32, %eax
@@ -465,12 +470,13 @@ define i32 @rotl_with_fshl_maybe_zero(i32 %x, i32 %y) {
; CHECK-LABEL: rotl_with_fshl_maybe_zero:
; CHECK: # %bb.0:
; CHECK-NEXT: movl %esi, %ecx
+; CHECK-NEXT: movl %edi, %eax
; CHECK-NEXT: # kill: def $cl killed $cl killed $ecx
-; CHECK-NEXT: roll %cl, %edi
+; CHECK-NEXT: roll %cl, %eax
; CHECK-NEXT: testl %edi, %edi
; CHECK-NEXT: je .LBB25_1
; CHECK-NEXT: # %bb.2: # %cond.false
-; CHECK-NEXT: rep bsfl %edi, %eax
+; CHECK-NEXT: rep bsfl %eax, %eax
; CHECK-NEXT: retq
; CHECK-NEXT: .LBB25_1:
; CHECK-NEXT: movl $32, %eax
diff --git a/llvm/test/CodeGen/X86/midpoint-int-vec-128.ll b/llvm/test/CodeGen/X86/midpoint-int-vec-128.ll
index d3cced3233ea65..5a6375e08bcaff 100644
--- a/llvm/test/CodeGen/X86/midpoint-int-vec-128.ll
+++ b/llvm/test/CodeGen/X86/midpoint-int-vec-128.ll
@@ -893,27 +893,26 @@ define <2 x i64> @vec128_i64_signed_reg_reg(<2 x i64> %a1, <2 x i64> %a2) nounwi
; SSE41-NEXT: pcmpeqd %xmm3, %xmm0
; SSE41-NEXT: pshufd {{.*#+}} xmm3 = xmm0[1,1,3,3]
; SSE41-NEXT: pand %xmm5, %xmm3
-; SSE41-NEXT: pshufd {{.*#+}} xmm6 = xmm4[1,1,3,3]
-; SSE41-NEXT: por %xmm3, %xmm6
-; SSE41-NEXT: por {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm6
-; SSE41-NEXT: pand %xmm5, %xmm0
-; SSE41-NEXT: por %xmm4, %xmm0
-; SSE41-NEXT: movdqa %xmm2, %xmm3
-; SSE41-NEXT: psubq %xmm1, %xmm3
+; SSE41-NEXT: pshufd {{.*#+}} xmm0 = xmm4[1,1,3,3]
+; SSE41-NEXT: por %xmm3, %xmm0
+; SSE41-NEXT: pmovsxbq {{.*#+}} xmm3 = [1,1]
+; SSE41-NEXT: por %xmm0, %xmm3
+; SSE41-NEXT: movdqa %xmm2, %xmm4
+; SSE41-NEXT: psubq %xmm1, %xmm4
; SSE41-NEXT: psubq %xmm2, %xmm1
-; SSE41-NEXT: blendvpd %xmm0, %xmm3, %xmm1
+; SSE41-NEXT: blendvpd %xmm0, %xmm4, %xmm1
; SSE41-NEXT: movapd %xmm1, %xmm0
; SSE41-NEXT: psrlq $1, %xmm0
; SSE41-NEXT: psrlq $33, %xmm1
-; SSE41-NEXT: pmuludq %xmm6, %xmm1
-; SSE41-NEXT: movdqa %xmm6, %xmm3
-; SSE41-NEXT: psrlq $32, %xmm3
-; SSE41-NEXT: pmuludq %xmm0, %xmm3
-; SSE41-NEXT: paddq %xmm1, %xmm3
-; SSE41-NEXT: psllq $32, %xmm3
-; SSE41-NEXT: pmuludq %xmm6, %xmm0
+; SSE41-NEXT: pmuludq %xmm3, %xmm1
+; SSE41-NEXT: movdqa %xmm3, %xmm4
+; SSE41-NEXT: psrlq $32, %xmm4
+; SSE41-NEXT: pmuludq %xmm0, %xmm4
+; SSE41-NEXT: paddq %xmm1, %xmm4
+; SSE41-NEXT: psllq $32, %xmm4
+; SSE41-NEXT: pmuludq %xmm3, %xmm0
; SSE41-NEXT: paddq %xmm2, %xmm0
-; SSE41-NEXT: paddq %xmm3, %xmm0
+; SSE41-NEXT: paddq %xmm4, %xmm0
; SSE41-NEXT: retq
;
; AVX-LABEL: vec128_i64_signed_reg_reg:
@@ -1077,27 +1076,26 @@ define <2 x i64> @vec128_i64_unsigned_reg_reg(<2 x i64> %a1, <2 x i64> %a2) noun
; SSE41-NEXT: pcmpeqd %xmm3, %xmm0
; SSE41-NEXT: pshufd {{.*#+}} xmm3 = xmm0[1,1,3,3]
; SSE41-NEXT: pand %xmm5, %xmm3
-; SSE41-NEXT: pshufd {{.*#+}} xmm6 = xmm4[1,1,3,3]
-; SSE41-NEXT: por %xmm3, %xmm6
-; SSE41-NEXT: por {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm6
-; SSE41-NEXT: pand %xmm5, %xmm0
-; SSE41-NEXT: por %xmm4, %xmm0
-; SSE41-NEXT: movdqa %xmm2, %xmm3
-; SSE41-NEXT: psubq %xmm1, %xmm3
+; SSE41-NEXT: pshufd {{.*#+}} xmm0 = xmm4[1,1,3,3]
+; SSE41-NEXT: por %xmm3, %xmm0
+; SSE41-NEXT: pmovsxbq {{.*#+}} xmm3 = [1,1]
+; SSE41-NEXT: por %xmm0, %xmm3
+; SSE41-NEXT: movdqa %xmm2, %xmm4
+; SSE41-NEXT: psubq %xmm1, %xmm4
; SSE41-NEXT: psubq %xmm2, %xmm1
-; SSE41-NEXT: blendvpd %xmm0, %xmm3, %xmm1
+; SSE41-NEXT: blendvpd %xmm0, %xmm4, %xmm1
; SSE41-NEXT: movapd %xmm1, %xmm0
; SSE41-NEXT: psrlq $1, %xmm0
; SSE41-NEXT: psrlq $33, %xmm1
-; SSE41-NEXT: pmuludq %xmm6, %xmm1
-; SSE41-NEXT: movdqa %xmm6, %xmm3
-; SSE41-NEXT: psrlq $32, %xmm3
-; SSE41-NEXT: pmuludq %xmm0, %xmm3
-; SSE41-NEXT: paddq %xmm1, %xmm3
-; SSE41-NEXT: psllq $32, %xmm3
-; SSE41-NEXT: pmuludq %xmm6, %xmm0
+; SSE41-NEXT: pmuludq %xmm3, %xmm1
+; SSE41-NEXT: movdqa %xmm3, %xmm4
+; SSE41-NEXT: psrlq $32, %xmm4
+; SSE41-NEXT: pmuludq %xmm0, %xmm4
+; SSE41-NEXT: paddq %xmm1, %xmm4
+; SSE41-NEXT: psllq $32, %xmm4
+; SSE41-NEXT: pmuludq %xmm3, %xmm0
; SSE41-NEXT: paddq %xmm2, %xmm0
-; SSE41-NEXT: paddq %xmm3, %xmm0
+; SSE41-NEXT: paddq %xmm4, %xmm0
; SSE41-NEXT: retq
;
; AVX1-LABEL: vec128_i64_unsigned_reg_reg:
@@ -1993,14 +1991,14 @@ define <8 x i16> @vec128_i16_unsigned_reg_reg(<8 x i16> %a1, <8 x i16> %a2) noun
;
; AVX512VL-FALLBACK-LABEL: vec128_i16_unsigned_reg_reg:
; AVX512VL-FALLBACK: # %bb.0:
-; AVX512VL-FALLBACK-NEXT: vpminuw %xmm1, %xmm0, %xmm2
-; AVX512VL-FALLBACK-NEXT: vpmaxuw %xmm1, %xmm0, %xmm1
-; AVX512VL-FALLBACK-NEXT: vpsubw %xmm2, %xmm1, %xmm1
-; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %xmm1, %xmm1
-; AVX512VL-FALLBACK-NEXT: vpcmpeqw %xmm2, %xmm0, %xmm2
-; AVX512VL-FALLBACK-NEXT: vpternlogq $15, %xmm2, %xmm2, %xmm2
-; AVX512VL-FALLBACK-NEXT: vpxor %xmm2, %xmm1, %xmm1
-; AVX512VL-FALLBACK-NEXT: vpsubw %xmm2, %xmm1, %xmm1
+; AVX512VL-FALLBACK-NEXT: vpmaxuw %xmm1, %xmm0, %xmm2
+; AVX512VL-FALLBACK-NEXT: vpminuw %xmm1, %xmm0, %xmm1
+; AVX512VL-FALLBACK-NEXT: vpsubw %xmm1, %xmm2, %xmm2
+; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %xmm2, %xmm2
+; AVX512VL-FALLBACK-NEXT: vpcmpeqw %xmm1, %xmm0, %xmm1
+; AVX512VL-FALLBACK-NEXT: vpternlogq $15, %xmm1, %xmm1, %xmm1
+; AVX512VL-FALLBACK-NEXT: vpxor %xmm1, %xmm2, %xmm2
+; AVX512VL-FALLBACK-NEXT: vpsubw %xmm1, %xmm2, %xmm1
; AVX512VL-FALLBACK-NEXT: vpaddw %xmm0, %xmm1, %xmm0
; AVX512VL-FALLBACK-NEXT: retq
;
@@ -2786,14 +2784,14 @@ define <16 x i8> @vec128_i8_unsigned_reg_reg(<16 x i8> %a1, <16 x i8> %a2) nounw
;
; AVX512VL-FALLBACK-LABEL: vec128_i8_unsigned_reg_reg:
; AVX512VL-FALLBACK: # %bb.0:
-; AVX512VL-FALLBACK-NEXT: vpminub %xmm1, %xmm0, %xmm2
-; AVX512VL-FALLBACK-NEXT: vpmaxub %xmm1, %xmm0, %xmm1
-; AVX512VL-FALLBACK-NEXT: vpsubb %xmm2, %xmm1, %xmm1
-; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %xmm1, %xmm1
-; AVX512VL-FALLBACK-NEXT: vpcmpeqb %xmm2, %xmm0, %xmm2
-; AVX512VL-FALLBACK-NEXT: vpternlogq $15, %xmm2, %xmm2, %xmm2
-; AVX512VL-FALLBACK-NEXT: vpternlogd $108, {{\.?LCPI[0-9]+_[0-9]+}}(%rip){1to4}, %xmm2, %xmm1
-; AVX512VL-FALLBACK-NEXT: vpsubb %xmm2, %xmm1, %xmm1
+; AVX512VL-FALLBACK-NEXT: vpmaxub %xmm1, %xmm0, %xmm2
+; AVX512VL-FALLBACK-NEXT: vpminub %xmm1, %xmm0, %xmm1
+; AVX512VL-FALLBACK-NEXT: vpsubb %xmm1, %xmm2, %xmm2
+; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %xmm2, %xmm2
+; AVX512VL-FALLBACK-NEXT: vpcmpeqb %xmm1, %xmm0, %xmm1
+; AVX512VL-FALLBACK-NEXT: vpternlogq $15, %xmm1, %xmm1, %xmm1
+; AVX512VL-FALLBACK-NEXT: vpternlogd $108, {{\.?LCPI[0-9]+_[0-9]+}}(%rip){1to4}, %xmm1, %xmm2
+; AVX512VL-FALLBACK-NEXT: vpsubb %xmm1, %xmm2, %xmm1
; AVX512VL-FALLBACK-NEXT: vpaddb %xmm0, %xmm1, %xmm0
; AVX512VL-FALLBACK-NEXT: retq
;
diff --git a/llvm/test/CodeGen/X86/midpoint-int-vec-256.ll b/llvm/test/CodeGen/X86/midpoint-int-vec-256.ll
index cc08396ae8c78f..e880a1acc9e83f 100644
--- a/llvm/test/CodeGen/X86/midpoint-int-vec-256.ll
+++ b/llvm/test/CodeGen/X86/midpoint-int-vec-256.ll
@@ -1445,14 +1445,14 @@ define <16 x i16> @vec256_i16_unsigned_reg_reg(<16 x i16> %a1, <16 x i16> %a2) n
;
; AVX512VL-FALLBACK-LABEL: vec256_i16_unsigned_reg_reg:
; AVX512VL-FALLBACK: # %bb.0:
-; AVX512VL-FALLBACK-NEXT: vpminuw %ymm1, %ymm0, %ymm2
-; AVX512VL-FALLBACK-NEXT: vpmaxuw %ymm1, %ymm0, %ymm1
-; AVX512VL-FALLBACK-NEXT: vpsubw %ymm2, %ymm1, %ymm1
-; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
-; AVX512VL-FALLBACK-NEXT: vpcmpeqw %ymm2, %ymm0, %ymm2
-; AVX512VL-FALLBACK-NEXT: vpternlogq $15, %ymm2, %ymm2, %ymm2
-; AVX512VL-FALLBACK-NEXT: vpxor %ymm2, %ymm1, %ymm1
-; AVX512VL-FALLBACK-NEXT: vpsubw %ymm2, %ymm1, %ymm1
+; AVX512VL-FALLBACK-NEXT: vpmaxuw %ymm1, %ymm0, %ymm2
+; AVX512VL-FALLBACK-NEXT: vpminuw %ymm1, %ymm0, %ymm1
+; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm2, %ymm2
+; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm2
+; AVX512VL-FALLBACK-NEXT: vpcmpeqw %ymm1, %ymm0, %ymm1
+; AVX512VL-FALLBACK-NEXT: vpternlogq $15, %ymm1, %ymm1, %ymm1
+; AVX512VL-FALLBACK-NEXT: vpxor %ymm1, %ymm2, %ymm2
+; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm2, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm0, %ymm1, %ymm0
; AVX512VL-FALLBACK-NEXT: retq
;
@@ -2210,14 +2210,14 @@ define <32 x i8> @vec256_i8_unsigned_reg_reg(<32 x i8> %a1, <32 x i8> %a2) nounw
;
; AVX512VL-FALLBACK-LABEL: vec256_i8_unsigned_reg_reg:
; AVX512VL-FALLBACK: # %bb.0:
-; AVX512VL-FALLBACK-NEXT: vpminub %ymm1, %ymm0, %ymm2
-; AVX512VL-FALLBACK-NEXT: vpmaxub %ymm1, %ymm0, %ymm1
-; AVX512VL-FALLBACK-NEXT: vpsubb %ymm2, %ymm1, %ymm1
-; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
-; AVX512VL-FALLBACK-NEXT: vpcmpeqb %ymm2, %ymm0, %ymm2
-; AVX512VL-FALLBACK-NEXT: vpternlogq $15, %ymm2, %ymm2, %ymm2
-; AVX512VL-FALLBACK-NEXT: vpternlogd $108, {{\.?LCPI[0-9]+_[0-9]+}}(%rip){1to8}, %ymm2, %ymm1
-; AVX512VL-FALLBACK-NEXT: vpsubb %ymm2, %ymm1, %ymm1
+; AVX512VL-FALLBACK-NEXT: vpmaxub %ymm1, %ymm0, %ymm2
+; AVX512VL-FALLBACK-NEXT: vpminub %ymm1, %ymm0, %ymm1
+; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm2, %ymm2
+; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm2
+; AVX512VL-FALLBACK-NEXT: vpcmpeqb %ymm1, %ymm0, %ymm1
+; AVX512VL-FALLBACK-NEXT: vpternlogq $15, %ymm1, %ymm1, %ymm1
+; AVX512VL-FALLBACK-NEXT: vpternlogd $108, {{\.?LCPI[0-9]+_[0-9]+}}(%rip){1to8}, %ymm1, %ymm2
+; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm2, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm0, %ymm1, %ymm0
; AVX512VL-FALLBACK-NEXT: retq
;
diff --git a/llvm/test/CodeGen/X86/pr38539.ll b/llvm/test/CodeGen/X86/pr38539.ll
index 04aff9b7d2e586..ace78b38d53edb 100644
--- a/llvm/test/CodeGen/X86/pr38539.ll
+++ b/llvm/test/CodeGen/X86/pr38539.ll
@@ -22,7 +22,7 @@ define void @f() nounwind {
; X86-NEXT: pushl %edi
; X86-NEXT: pushl %esi
; X86-NEXT: andl $-16, %esp
-; X86-NEXT: subl $176, %esp
+; X86-NEXT: subl $160, %esp
; X86-NEXT: movzbl {{[0-9]+}}(%esp), %edx
; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
; X86-NEXT: movl {{[0-9]+}}(%esp), %edi
@@ -47,55 +47,54 @@ define void @f() nounwind {
; X86-NEXT: testl %edi, %edi
; X86-NEXT: jne .LBB0_1
; X86-NEXT: # %bb.2: # %BB_udiv-special-cases
-; X86-NEXT: bsrl %esi, %ecx
-; X86-NEXT: xorl $31, %ecx
-; X86-NEXT: addl $32, %ecx
+; X86-NEXT: bsrl %esi, %eax
+; X86-NEXT: xorl $31, %eax
+; X86-NEXT: addl $32, %eax
; X86-NEXT: jmp .LBB0_3
; X86-NEXT: .LBB0_1:
-; X86-NEXT: bsrl %edi, %ecx
-; X86-NEXT: xorl $31, %ecx
+; X86-NEXT: bsrl %edi, %eax
+; X86-NEXT: xorl $31, %eax
; X86-NEXT: .LBB0_3: # %BB_udiv-special-cases
-; X86-NEXT: xorl %eax, %eax
+; X86-NEXT: xorl %ecx, %ecx
; X86-NEXT: testl %edx, %edx
-; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: jne .LBB0_4
; X86-NEXT: # %bb.5: # %BB_udiv-special-cases
-; X86-NEXT: addl $64, %ecx
+; X86-NEXT: addl $64, %eax
; X86-NEXT: jmp .LBB0_6
; X86-NEXT: .LBB0_4:
-; X86-NEXT: bsrl %edx, %ecx
-; X86-NEXT: xorl $31, %ecx
-; X86-NEXT: addl $32, %ecx
+; X86-NEXT: bsrl %edx, %eax
+; X86-NEXT: xorl $31, %eax
+; X86-NEXT: addl $32, %eax
; X86-NEXT: .LBB0_6: # %BB_udiv-special-cases
-; X86-NEXT: subl $62, %ecx
+; X86-NEXT: subl $62, %eax
; X86-NEXT: movl $0, %ebx
; X86-NEXT: sbbl %ebx, %ebx
-; X86-NEXT: sbbl %eax, %eax
-; X86-NEXT: addl $-66, %ecx
+; X86-NEXT: sbbl %ecx, %ecx
+; X86-NEXT: addl $-66, %eax
; X86-NEXT: adcl $-1, %ebx
-; X86-NEXT: adcl $3, %eax
-; X86-NEXT: movl %eax, %edi
-; X86-NEXT: movb $1, %al
-; X86-NEXT: testb %al, %al
+; X86-NEXT: adcl $3, %ecx
+; X86-NEXT: movl %ecx, %esi
+; X86-NEXT: movb $1, %cl
+; X86-NEXT: testb %cl, %cl
; X86-NEXT: jne .LBB0_11
; X86-NEXT: # %bb.7: # %BB_udiv-special-cases
-; X86-NEXT: andl $3, %edi
-; X86-NEXT: movl %ecx, %eax
-; X86-NEXT: xorl $65, %eax
-; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: orl %edi, %eax
-; X86-NEXT: orl %ebx, %eax
+; X86-NEXT: andl $3, %esi
+; X86-NEXT: movl %eax, %ecx
+; X86-NEXT: xorl $65, %ecx
+; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: orl %esi, %ecx
+; X86-NEXT: orl %ebx, %ecx
; X86-NEXT: je .LBB0_11
; X86-NEXT: # %bb.8: # %udiv-bb1
-; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %ecx, %eax
+; X86-NEXT: movl %eax, %ecx
; X86-NEXT: addl $1, %ecx
; X86-NEXT: movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: adcl $0, %ebx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT: adcl $0, %esi
-; X86-NEXT: andl $3, %esi
-; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT: adcl $0, %ebx
+; X86-NEXT: andl $3, %ebx
; X86-NEXT: movb $65, %cl
; X86-NEXT: subb %al, %cl
; X86-NEXT: movb %cl, %ch
@@ -112,29 +111,31 @@ define void @f() nounwind {
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
-; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl 136(%esp,%eax), %edx
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl 120(%esp,%eax), %edi
; X86-NEXT: movb %ch, %cl
-; X86-NEXT: shll %cl, %edx
+; X86-NEXT: shll %cl, %edi
; X86-NEXT: notb %cl
-; X86-NEXT: movl 128(%esp,%eax), %edi
-; X86-NEXT: movl 132(%esp,%eax), %esi
-; X86-NEXT: movl %esi, %eax
+; X86-NEXT: movl 112(%esp,%eax), %esi
+; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl 116(%esp,%eax), %edx
+; X86-NEXT: movl %edx, %eax
; X86-NEXT: shrl %eax
; X86-NEXT: shrl %cl, %eax
; X86-NEXT: movb %ch, %cl
-; X86-NEXT: shldl %cl, %edi, %esi
+; X86-NEXT: shldl %cl, %esi, %edx
+; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: shll %cl, %esi
; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: shll %cl, %edi
-; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
+; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: orl %ebx, %ecx
+; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
; X86-NEXT: je .LBB0_11
; X86-NEXT: # %bb.9: # %udiv-preheader
-; X86-NEXT: orl %eax, %edx
-; X86-NEXT: andl $3, %edx
-; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: orl %eax, %edi
+; X86-NEXT: andl $3, %edi
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
; X86-NEXT: movl $0, {{[0-9]+}}(%esp)
@@ -149,20 +150,20 @@ define void @f() nounwind {
; X86-NEXT: # kill: def $al killed $al killed $eax
; X86-NEXT: shrb $3, %al
; X86-NEXT: andb $15, %al
-; X86-NEXT: movzbl %al, %esi
-; X86-NEXT: movl 80(%esp,%esi), %edx
-; X86-NEXT: movl 84(%esp,%esi), %eax
-; X86-NEXT: movl %eax, %edi
+; X86-NEXT: movzbl %al, %eax
+; X86-NEXT: movl 64(%esp,%eax), %edi
+; X86-NEXT: movl 68(%esp,%eax), %edx
+; X86-NEXT: movl %edx, %esi
; X86-NEXT: movb %ch, %cl
-; X86-NEXT: shrl %cl, %edi
+; X86-NEXT: shrl %cl, %esi
; X86-NEXT: notb %cl
-; X86-NEXT: movl 88(%esp,%esi), %esi
-; X86-NEXT: addl %esi, %esi
-; X86-NEXT: shll %cl, %esi
-; X86-NEXT: orl %edi, %esi
+; X86-NEXT: movl 72(%esp,%eax), %ebx
+; X86-NEXT: addl %ebx, %ebx
+; X86-NEXT: shll %cl, %ebx
+; X86-NEXT: orl %esi, %ebx
; X86-NEXT: movb %ch, %cl
-; X86-NEXT: shrdl %cl, %eax, %edx
-; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: shrdl %cl, %edx, %edi
+; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: addl $-1, %eax
@@ -179,63 +180,62 @@ define void @f() nounwind {
; X86-NEXT: .p2align 4, 0x90
; X86-NEXT: .LBB0_10: # %udiv-do-while
; X86-NEXT: # =>This Inner Loop Header: Depth=1
-; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: shldl $1, %esi, %ecx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT: shldl $1, %edx, %esi
-; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: movl %ebx, %edi
+; X86-NEXT: shldl $1, %ebx, %ecx
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT: movl %ebx, %eax
-; X86-NEXT: andl $2, %eax
-; X86-NEXT: shrl %eax
-; X86-NEXT: leal (%eax,%edx,2), %edx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT: shldl $1, %edi, %ebx
+; X86-NEXT: shldl $1, %ebx, %edi
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT: orl %esi, %ebx
+; X86-NEXT: movl %esi, %edx
+; X86-NEXT: andl $2, %edx
+; X86-NEXT: shrl %edx
+; X86-NEXT: leal (%edx,%ebx,2), %ebx
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT: shldl $1, %edx, %esi
+; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT: shldl $1, %eax, %edi
-; X86-NEXT: orl %esi, %edi
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: shldl $1, %eax, %edx
+; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
+; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
; X86-NEXT: addl %eax, %eax
; X86-NEXT: orl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: andl $3, %ebx
-; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: cmpl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT: sbbl %esi, %eax
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT: sbbl %ecx, %ebx
-; X86-NEXT: shll $30, %ebx
-; X86-NEXT: movl %ebx, %eax
-; X86-NEXT: sarl $30, %eax
-; X86-NEXT: sarl $31, %ebx
-; X86-NEXT: shrdl $1, %ebx, %eax
-; X86-NEXT: movl %eax, %edi
-; X86-NEXT: andl $1, %edi
-; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: andl $3, %esi
+; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: cmpl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT: sbbl %edi, %edx
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT: sbbl %ecx, %esi
+; X86-NEXT: shll $30, %esi
+; X86-NEXT: movl %esi, %edx
+; X86-NEXT: sarl $30, %edx
+; X86-NEXT: sarl $31, %esi
+; X86-NEXT: shrdl $1, %esi, %edx
+; X86-NEXT: movl %edx, %eax
+; X86-NEXT: andl $1, %eax
+; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: andl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
+; X86-NEXT: movl %esi, %eax
; X86-NEXT: andl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; X86-NEXT: movl %ebx, %edi
-; X86-NEXT: andl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
-; X86-NEXT: andl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
-; X86-NEXT: subl %eax, %edx
-; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: sbbl %ebx, %esi
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT: sbbl %edi, %ecx
+; X86-NEXT: andl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
+; X86-NEXT: subl %edx, %ebx
+; X86-NEXT: movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: sbbl %esi, %edi
+; X86-NEXT: movl %edi, %ebx
+; X86-NEXT: sbbl %eax, %ecx
; X86-NEXT: andl $3, %ecx
; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
; X86-NEXT: addl $-1, %eax
-; X86-NEXT: adcl $-1, %ebx
-; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT: adcl $3, %edi
-; X86-NEXT: andl $3, %edi
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT: adcl $-1, %edx
+; X86-NEXT: movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT: adcl $3, %esi
+; X86-NEXT: andl $3, %esi
; X86-NEXT: movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT: orl %edi, %eax
-; X86-NEXT: orl %ebx, %eax
+; X86-NEXT: movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: orl %esi, %eax
+; X86-NEXT: movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT: orl %edx, %eax
; X86-NEXT: jne .LBB0_10
; X86-NEXT: .LBB0_11: # %udiv-end
; X86-NEXT: cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
diff --git a/llvm/test/CodeGen/X86/pr62286.ll b/llvm/test/CodeGen/X86/pr62286.ll
index 782c84408f25ab..7ce8756fd9f9d9 100644
--- a/llvm/test/CodeGen/X86/pr62286.ll
+++ b/llvm/test/CodeGen/X86/pr62286.ll
@@ -8,21 +8,20 @@ define i64 @PR62286(i32 %a) {
; SSE-LABEL: PR62286:
; SSE: # %bb.0:
; SSE-NEXT: movd %edi, %xmm0
-; SSE-NEXT: pshufd {{.*#+}} xmm1 = xmm0[0,1,1,0]
-; SSE-NEXT: paddd %xmm1, %xmm1
+; SSE-NEXT: pshufd {{.*#+}} xmm1 = xmm0[1,0,1,0]
+; SSE-NEXT: paddd %xmm0, %xmm0
; SSE-NEXT: pxor %xmm2, %xmm2
-; SSE-NEXT: punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
-; SSE-NEXT: pxor %xmm3, %xmm3
-; SSE-NEXT: pcmpgtd %xmm1, %xmm3
-; SSE-NEXT: punpckldq {{.*#+}} xmm1 = xmm1[0],xmm3[0],xmm1[1],xmm3[1]
-; SSE-NEXT: pshufd {{.*#+}} xmm0 = xmm0[1,0,1,0]
-; SSE-NEXT: pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
-; SSE-NEXT: pcmpgtd %xmm0, %xmm2
; SSE-NEXT: punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
-; SSE-NEXT: paddq %xmm1, %xmm0
-; SSE-NEXT: pshufd {{.*#+}} xmm1 = xmm0[2,3,2,3]
+; SSE-NEXT: pxor %xmm3, %xmm3
+; SSE-NEXT: pcmpgtd %xmm0, %xmm3
+; SSE-NEXT: punpckldq {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[1],xmm3[1]
+; SSE-NEXT: pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm1
+; SSE-NEXT: pcmpgtd %xmm1, %xmm2
+; SSE-NEXT: punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
; SSE-NEXT: paddq %xmm0, %xmm1
-; SSE-NEXT: movq %xmm1, %rax
+; SSE-NEXT: pshufd {{.*#+}} xmm0 = xmm1[2,3,2,3]
+; SSE-NEXT: paddq %xmm1, %xmm0
+; SSE-NEXT: movq %xmm0, %rax
; SSE-NEXT: retq
;
; AVX1-LABEL: PR62286:
diff --git a/llvm/test/CodeGen/X86/scheduler-backtracking.ll b/llvm/test/CodeGen/X86/scheduler-backtracking.ll
index a9f3e8b22fb69e..785b97d8c24027 100644
--- a/llvm/test/CodeGen/X86/scheduler-backtracking.ll
+++ b/llvm/test/CodeGen/X86/scheduler-backtracking.ll
@@ -14,7 +14,6 @@ define i256 @test1(i256 %a) nounwind {
; ILP: # %bb.0:
; ILP-NEXT: movq %rdi, %rax
; ILP-NEXT: leal (%rsi,%rsi), %ecx
-; ILP-NEXT: addb $3, %cl
; ILP-NEXT: movq $0, -{{[0-9]+}}(%rsp)
; ILP-NEXT: movq $0, -{{[0-9]+}}(%rsp)
; ILP-NEXT: movq $0, -{{[0-9]+}}(%rsp)
@@ -23,6 +22,7 @@ define i256 @test1(i256 %a) nounwind {
; ILP-NEXT: movq $0, -{{[0-9]+}}(%rsp)
; ILP-NEXT: movq $0, -{{[0-9]+}}(%rsp)
; ILP-NEXT: movq $0, -{{[0-9]+}}(%rsp)
+; ILP-NEXT: addb $3, %cl
; ILP-NEXT: movl %ecx, %edx
; ILP-NEXT: shrb $3, %dl
; ILP-NEXT: andb $7, %cl
diff --git a/llvm/test/CodeGen/X86/setcc-non-simple-type.ll b/llvm/test/CodeGen/X86/setcc-non-simple-type.ll
index 2187c653f76c3b..e5451f1f99608b 100644
--- a/llvm/test/CodeGen/X86/setcc-non-simple-type.ll
+++ b/llvm/test/CodeGen/X86/setcc-non-simple-type.ll
@@ -46,7 +46,6 @@ define void @failing(ptr %0, ptr %1) nounwind {
; CHECK-NEXT: movq 24(%rsi), %rcx
; CHECK-NEXT: movq 32(%rsi), %rdx
; CHECK-NEXT: movdqa {{.*#+}} xmm0 = [0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0]
-; CHECK-NEXT: xorl %esi, %esi
; CHECK-NEXT: movdqa {{.*#+}} xmm1 = [1,1]
; CHECK-NEXT: movdqa {{.*#+}} xmm2 = [2,2]
; CHECK-NEXT: .p2align 4, 0x90
@@ -54,34 +53,34 @@ define void @failing(ptr %0, ptr %1) nounwind {
; CHECK-NEXT: # =>This Loop Header: Depth=1
; CHECK-NEXT: # Child Loop BB0_2 Depth 2
; CHECK-NEXT: xorpd %xmm3, %xmm3
-; CHECK-NEXT: movq $-1024, %rdi # imm = 0xFC00
+; CHECK-NEXT: movq $-1024, %rsi # imm = 0xFC00
; CHECK-NEXT: movdqa %xmm0, %xmm4
; CHECK-NEXT: .p2align 4, 0x90
; CHECK-NEXT: .LBB0_2: # %vector.body
; CHECK-NEXT: # Parent Loop BB0_1 Depth=1
; CHECK-NEXT: # => This Inner Loop Header: Depth=2
-; CHECK-NEXT: movdqu 1024(%rdx,%rdi), %xmm5
-; CHECK-NEXT: movdqu 1040(%rdx,%rdi), %xmm6
+; CHECK-NEXT: movq 1024(%rdx,%rsi), %rdi
+; CHECK-NEXT: movq 1040(%rdx,%rsi), %r8
+; CHECK-NEXT: movdqu 1024(%rdx,%rsi), %xmm5
+; CHECK-NEXT: movdqu 1040(%rdx,%rsi), %xmm6
+; CHECK-NEXT: pshufd {{.*#+}} xmm6 = xmm6[2,3,2,3]
+; CHECK-NEXT: movq %xmm6, %r9
; CHECK-NEXT: pshufd {{.*#+}} xmm5 = xmm5[2,3,2,3]
-; CHECK-NEXT: movq %xmm5, %r8
-; CHECK-NEXT: pshufd {{.*#+}} xmm5 = xmm6[2,3,2,3]
-; CHECK-NEXT: movq %xmm5, %r9
-; CHECK-NEXT: cmpq 1040(%rdx,%rdi), %rsi
-; CHECK-NEXT: movq %rcx, %r10
-; CHECK-NEXT: sbbq %r9, %r10
-; CHECK-NEXT: setge %r9b
-; CHECK-NEXT: movzbl %r9b, %r9d
-; CHECK-NEXT: andl $1, %r9d
-; CHECK-NEXT: negq %r9
-; CHECK-NEXT: movq %r9, %xmm5
-; CHECK-NEXT: cmpq 1024(%rdx,%rdi), %rsi
-; CHECK-NEXT: movq %rcx, %r9
-; CHECK-NEXT: sbbq %r8, %r9
+; CHECK-NEXT: movq %xmm5, %r10
+; CHECK-NEXT: negq %r8
+; CHECK-NEXT: movq %rcx, %r8
+; CHECK-NEXT: sbbq %r9, %r8
; CHECK-NEXT: setge %r8b
; CHECK-NEXT: movzbl %r8b, %r8d
-; CHECK-NEXT: andl $1, %r8d
; CHECK-NEXT: negq %r8
-; CHECK-NEXT: movq %r8, %xmm6
+; CHECK-NEXT: movq %r8, %xmm5
+; CHECK-NEXT: negq %rdi
+; CHECK-NEXT: movq %rcx, %rdi
+; CHECK-NEXT: sbbq %r10, %rdi
+; CHECK-NEXT: setge %dil
+; CHECK-NEXT: movzbl %dil, %edi
+; CHECK-NEXT: negq %rdi
+; CHECK-NEXT: movq %rdi, %xmm6
; CHECK-NEXT: punpcklqdq {{.*#+}} xmm6 = xmm6[0],xmm5[0]
; CHECK-NEXT: movdqa %xmm1, %xmm5
; CHECK-NEXT: psllq %xmm4, %xmm5
@@ -92,7 +91,7 @@ define void @failing(ptr %0, ptr %1) nounwind {
; CHECK-NEXT: andpd %xmm6, %xmm8
; CHECK-NEXT: orpd %xmm8, %xmm3
; CHECK-NEXT: paddq %xmm2, %xmm4
-; CHECK-NEXT: addq $32, %rdi
+; CHECK-NEXT: addq $32, %rsi
; CHECK-NEXT: jne .LBB0_2
; CHECK-NEXT: # %bb.3: # %middle.block
; CHECK-NEXT: # in Loop: Header=BB0_1 Depth=1
@@ -107,7 +106,6 @@ define void @failing(ptr %0, ptr %1) nounwind {
; CHECK-AVX2-NEXT: movq 24(%rsi), %rcx
; CHECK-AVX2-NEXT: movq 32(%rsi), %rdx
; CHECK-AVX2-NEXT: vpmovsxbq {{.*#+}} xmm0 = [0,1]
-; CHECK-AVX2-NEXT: xorl %esi, %esi
; CHECK-AVX2-NEXT: vpmovsxbq {{.*#+}} xmm1 = [1,1]
; CHECK-AVX2-NEXT: vpmovsxbq {{.*#+}} xmm2 = [2,2]
; CHECK-AVX2-NEXT: .p2align 4, 0x90
@@ -115,34 +113,34 @@ define void @failing(ptr %0, ptr %1) nounwind {
; CHECK-AVX2-NEXT: # =>This Loop Header: Depth=1
; CHECK-AVX2-NEXT: # Child Loop BB0_2 Depth 2
; CHECK-AVX2-NEXT: vpxor %xmm3, %xmm3, %xmm3
-; CHECK-AVX2-NEXT: movq $-1024, %rdi # imm = 0xFC00
+; CHECK-AVX2-NEXT: movq $-1024, %rsi # imm = 0xFC00
; CHECK-AVX2-NEXT: vmovdqa %xmm0, %xmm4
; CHECK-AVX2-NEXT: .p2align 4, 0x90
; CHECK-AVX2-NEXT: .LBB0_2: # %vector.body
; CHECK-AVX2-NEXT: # Parent Loop BB0_1 Depth=1
; CHECK-AVX2-NEXT: # => This Inner Loop Header: Depth=2
-; CHECK-AVX2-NEXT: cmpq 1024(%rdx,%rdi), %rsi
-; CHECK-AVX2-NEXT: movq %rcx, %r8
-; CHECK-AVX2-NEXT: sbbq 1032(%rdx,%rdi), %r8
-; CHECK-AVX2-NEXT: setge %r8b
-; CHECK-AVX2-NEXT: movzbl %r8b, %r8d
-; CHECK-AVX2-NEXT: andl $1, %r8d
-; CHECK-AVX2-NEXT: negq %r8
-; CHECK-AVX2-NEXT: vmovq %r8, %xmm5
-; CHECK-AVX2-NEXT: cmpq 1040(%rdx,%rdi), %rsi
-; CHECK-AVX2-NEXT: movq %rcx, %r8
-; CHECK-AVX2-NEXT: sbbq 1048(%rdx,%rdi), %r8
-; CHECK-AVX2-NEXT: setge %r8b
-; CHECK-AVX2-NEXT: movzbl %r8b, %r8d
-; CHECK-AVX2-NEXT: andl $1, %r8d
+; CHECK-AVX2-NEXT: movq 1040(%rdx,%rsi), %rdi
+; CHECK-AVX2-NEXT: movq 1024(%rdx,%rsi), %r8
+; CHECK-AVX2-NEXT: negq %rdi
+; CHECK-AVX2-NEXT: movq %rcx, %rdi
+; CHECK-AVX2-NEXT: sbbq 1048(%rdx,%rsi), %rdi
+; CHECK-AVX2-NEXT: setge %dil
+; CHECK-AVX2-NEXT: movzbl %dil, %edi
+; CHECK-AVX2-NEXT: negq %rdi
+; CHECK-AVX2-NEXT: vmovq %rdi, %xmm5
; CHECK-AVX2-NEXT: negq %r8
-; CHECK-AVX2-NEXT: vmovq %r8, %xmm6
-; CHECK-AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm5 = xmm5[0],xmm6[0]
+; CHECK-AVX2-NEXT: movq %rcx, %rdi
+; CHECK-AVX2-NEXT: sbbq 1032(%rdx,%rsi), %rdi
+; CHECK-AVX2-NEXT: setge %dil
+; CHECK-AVX2-NEXT: movzbl %dil, %edi
+; CHECK-AVX2-NEXT: negq %rdi
+; CHECK-AVX2-NEXT: vmovq %rdi, %xmm6
+; CHECK-AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm5 = xmm6[0],xmm5[0]
; CHECK-AVX2-NEXT: vpsllvq %xmm4, %xmm1, %xmm6
; CHECK-AVX2-NEXT: vpand %xmm6, %xmm5, %xmm5
; CHECK-AVX2-NEXT: vpor %xmm3, %xmm5, %xmm3
; CHECK-AVX2-NEXT: vpaddq %xmm2, %xmm4, %xmm4
-; CHECK-AVX2-NEXT: addq $32, %rdi
+; CHECK-AVX2-NEXT: addq $32, %rsi
; CHECK-AVX2-NEXT: jne .LBB0_2
; CHECK-AVX2-NEXT: # %bb.3: # %middle.block
; CHECK-AVX2-NEXT: # in Loop: Header=BB0_1 Depth=1
diff --git a/llvm/test/CodeGen/X86/vec_saddo.ll b/llvm/test/CodeGen/X86/vec_saddo.ll
index cee30f5fe5da9e..460c5fe11f82a5 100644
--- a/llvm/test/CodeGen/X86/vec_saddo.ll
+++ b/llvm/test/CodeGen/X86/vec_saddo.ll
@@ -1045,16 +1045,12 @@ define <4 x i32> @saddo_v4i1(<4 x i1> %a0, <4 x i1> %a1, ptr %p2) nounwind {
;
; AVX512-LABEL: saddo_v4i1:
; AVX512: # %bb.0:
+; AVX512-NEXT: vpxor %xmm1, %xmm0, %xmm2
+; AVX512-NEXT: vpslld $31, %xmm2, %xmm2
+; AVX512-NEXT: vptestmd %xmm2, %xmm2, %k0
+; AVX512-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpslld $31, %xmm0, %xmm0
-; AVX512-NEXT: vptestmd %xmm0, %xmm0, %k0
-; AVX512-NEXT: vpslld $31, %xmm1, %xmm0
-; AVX512-NEXT: vptestmd %xmm0, %xmm0, %k1
-; AVX512-NEXT: kxorw %k1, %k0, %k2
-; AVX512-NEXT: kandw %k1, %k0, %k1
-; AVX512-NEXT: vpcmpeqd %xmm0, %xmm0, %xmm0
-; AVX512-NEXT: vmovdqa32 %xmm0, %xmm0 {%k1} {z}
-; AVX512-NEXT: kshiftlw $12, %k2, %k0
-; AVX512-NEXT: kshiftrw $12, %k0, %k0
+; AVX512-NEXT: vpsrad $31, %xmm0, %xmm0
; AVX512-NEXT: kmovd %k0, %eax
; AVX512-NEXT: movb %al, (%rdi)
; AVX512-NEXT: retq
diff --git a/llvm/test/CodeGen/X86/vec_ssubo.ll b/llvm/test/CodeGen/X86/vec_ssubo.ll
index 64ed081048851b..d06993da6365d8 100644
--- a/llvm/test/CodeGen/X86/vec_ssubo.ll
+++ b/llvm/test/CodeGen/X86/vec_ssubo.ll
@@ -1062,16 +1062,12 @@ define <4 x i32> @ssubo_v4i1(<4 x i1> %a0, <4 x i1> %a1, ptr %p2) nounwind {
;
; AVX512-LABEL: ssubo_v4i1:
; AVX512: # %bb.0:
+; AVX512-NEXT: vpxor %xmm1, %xmm0, %xmm2
+; AVX512-NEXT: vpslld $31, %xmm2, %xmm2
+; AVX512-NEXT: vptestmd %xmm2, %xmm2, %k0
+; AVX512-NEXT: vpandn %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpslld $31, %xmm0, %xmm0
-; AVX512-NEXT: vptestmd %xmm0, %xmm0, %k0
-; AVX512-NEXT: vpslld $31, %xmm1, %xmm1
-; AVX512-NEXT: vptestmd %xmm1, %xmm1, %k1
-; AVX512-NEXT: kxorw %k1, %k0, %k0
-; AVX512-NEXT: vptestnmd %xmm0, %xmm0, %k1 {%k1}
-; AVX512-NEXT: vpcmpeqd %xmm0, %xmm0, %xmm0
-; AVX512-NEXT: vmovdqa32 %xmm0, %xmm0 {%k1} {z}
-; AVX512-NEXT: kshiftlw $12, %k0, %k0
-; AVX512-NEXT: kshiftrw $12, %k0, %k0
+; AVX512-NEXT: vpsrad $31, %xmm0, %xmm0
; AVX512-NEXT: kmovd %k0, %eax
; AVX512-NEXT: movb %al, (%rdi)
; AVX512-NEXT: retq
diff --git a/llvm/test/CodeGen/X86/vec_uaddo.ll b/llvm/test/CodeGen/X86/vec_uaddo.ll
index 950e943bd90201..bac118095331ca 100644
--- a/llvm/test/CodeGen/X86/vec_uaddo.ll
+++ b/llvm/test/CodeGen/X86/vec_uaddo.ll
@@ -1098,16 +1098,12 @@ define <4 x i32> @uaddo_v4i1(<4 x i1> %a0, <4 x i1> %a1, ptr %p2) nounwind {
;
; AVX512-LABEL: uaddo_v4i1:
; AVX512: # %bb.0:
+; AVX512-NEXT: vpxor %xmm1, %xmm0, %xmm2
+; AVX512-NEXT: vpslld $31, %xmm2, %xmm2
+; AVX512-NEXT: vptestmd %xmm2, %xmm2, %k0
+; AVX512-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpslld $31, %xmm0, %xmm0
-; AVX512-NEXT: vptestmd %xmm0, %xmm0, %k0
-; AVX512-NEXT: vpslld $31, %xmm1, %xmm0
-; AVX512-NEXT: vptestmd %xmm0, %xmm0, %k1
-; AVX512-NEXT: kxorw %k1, %k0, %k2
-; AVX512-NEXT: kandw %k1, %k0, %k1
-; AVX512-NEXT: vpcmpeqd %xmm0, %xmm0, %xmm0
-; AVX512-NEXT: vmovdqa32 %xmm0, %xmm0 {%k1} {z}
-; AVX512-NEXT: kshiftlw $12, %k2, %k0
-; AVX512-NEXT: kshiftrw $12, %k0, %k0
+; AVX512-NEXT: vpsrad $31, %xmm0, %xmm0
; AVX512-NEXT: kmovd %k0, %eax
; AVX512-NEXT: movb %al, (%rdi)
; AVX512-NEXT: retq
diff --git a/llvm/test/CodeGen/X86/vec_usubo.ll b/llvm/test/CodeGen/X86/vec_usubo.ll
index 7de972770d8da4..ab75ada72f2565 100644
--- a/llvm/test/CodeGen/X86/vec_usubo.ll
+++ b/llvm/test/CodeGen/X86/vec_usubo.ll
@@ -1145,16 +1145,12 @@ define <4 x i32> @usubo_v4i1(<4 x i1> %a0, <4 x i1> %a1, ptr %p2) nounwind {
;
; AVX512-LABEL: usubo_v4i1:
; AVX512: # %bb.0:
+; AVX512-NEXT: vpxor %xmm1, %xmm0, %xmm2
+; AVX512-NEXT: vpslld $31, %xmm2, %xmm2
+; AVX512-NEXT: vptestmd %xmm2, %xmm2, %k0
+; AVX512-NEXT: vpandn %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpslld $31, %xmm0, %xmm0
-; AVX512-NEXT: vptestmd %xmm0, %xmm0, %k0
-; AVX512-NEXT: vpslld $31, %xmm1, %xmm1
-; AVX512-NEXT: vptestmd %xmm1, %xmm1, %k1
-; AVX512-NEXT: kxorw %k1, %k0, %k0
-; AVX512-NEXT: vptestnmd %xmm0, %xmm0, %k1 {%k1}
-; AVX512-NEXT: vpcmpeqd %xmm0, %xmm0, %xmm0
-; AVX512-NEXT: vmovdqa32 %xmm0, %xmm0 {%k1} {z}
-; AVX512-NEXT: kshiftlw $12, %k0, %k0
-; AVX512-NEXT: kshiftrw $12, %k0, %k0
+; AVX512-NEXT: vpsrad $31, %xmm0, %xmm0
; AVX512-NEXT: kmovd %k0, %eax
; AVX512-NEXT: movb %al, (%rdi)
; AVX512-NEXT: retq
diff --git a/llvm/test/CodeGen/X86/vector-bo-select.ll b/llvm/test/CodeGen/X86/vector-bo-select.ll
index 78797b9acc2e6e..27aaad6353ed68 100644
--- a/llvm/test/CodeGen/X86/vector-bo-select.ll
+++ b/llvm/test/CodeGen/X86/vector-bo-select.ll
@@ -3137,11 +3137,11 @@ define <8 x i64> @mul_v8i64_cast_cond(i8 noundef zeroext %pb, <8 x i64> noundef
; AVX512-LABEL: mul_v8i64_cast_cond:
; AVX512: # %bb.0:
; AVX512-NEXT: kmovw %edi, %k1
-; AVX512-NEXT: vpsrlq $32, %zmm1, %zmm2
-; AVX512-NEXT: vpmuludq %zmm2, %zmm0, %zmm2
-; AVX512-NEXT: vpsrlq $32, %zmm0, %zmm3
-; AVX512-NEXT: vpmuludq %zmm1, %zmm3, %zmm3
-; AVX512-NEXT: vpaddq %zmm3, %zmm2, %zmm2
+; AVX512-NEXT: vpsrlq $32, %zmm0, %zmm2
+; AVX512-NEXT: vpmuludq %zmm1, %zmm2, %zmm2
+; AVX512-NEXT: vpsrlq $32, %zmm1, %zmm3
+; AVX512-NEXT: vpmuludq %zmm3, %zmm0, %zmm3
+; AVX512-NEXT: vpaddq %zmm2, %zmm3, %zmm2
; AVX512-NEXT: vpsllq $32, %zmm2, %zmm2
; AVX512-NEXT: vpmuludq %zmm1, %zmm0, %zmm1
; AVX512-NEXT: vpaddq %zmm2, %zmm1, %zmm0 {%k1}
diff --git a/llvm/test/CodeGen/X86/vector-fshr-128.ll b/llvm/test/CodeGen/X86/vector-fshr-128.ll
index 638a3cdaa2c1d2..1552e8a2340fce 100644
--- a/llvm/test/CodeGen/X86/vector-fshr-128.ll
+++ b/llvm/test/CodeGen/X86/vector-fshr-128.ll
@@ -58,12 +58,12 @@ define <2 x i64> @var_funnnel_v2i64(<2 x i64> %x, <2 x i64> %y, <2 x i64> %amt)
; SSE41-NEXT: psrlq %xmm4, %xmm1
; SSE41-NEXT: pblendw {{.*#+}} xmm5 = xmm5[0,1,2,3],xmm1[4,5,6,7]
; SSE41-NEXT: pandn %xmm3, %xmm2
-; SSE41-NEXT: pshufd {{.*#+}} xmm1 = xmm2[2,3,2,3]
; SSE41-NEXT: paddq %xmm0, %xmm0
-; SSE41-NEXT: movdqa %xmm0, %xmm3
-; SSE41-NEXT: psllq %xmm1, %xmm3
+; SSE41-NEXT: movdqa %xmm0, %xmm1
+; SSE41-NEXT: psllq %xmm2, %xmm1
+; SSE41-NEXT: pshufd {{.*#+}} xmm2 = xmm2[2,3,2,3]
; SSE41-NEXT: psllq %xmm2, %xmm0
-; SSE41-NEXT: pblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm3[4,5,6,7]
+; SSE41-NEXT: pblendw {{.*#+}} xmm0 = xmm1[0,1,2,3],xmm0[4,5,6,7]
; SSE41-NEXT: por %xmm5, %xmm0
; SSE41-NEXT: retq
;
@@ -76,11 +76,11 @@ define <2 x i64> @var_funnnel_v2i64(<2 x i64> %x, <2 x i64> %y, <2 x i64> %amt)
; AVX1-NEXT: vpsrlq %xmm4, %xmm1, %xmm1
; AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm5[0,1,2,3],xmm1[4,5,6,7]
; AVX1-NEXT: vpandn %xmm3, %xmm2, %xmm2
-; AVX1-NEXT: vpshufd {{.*#+}} xmm3 = xmm2[2,3,2,3]
; AVX1-NEXT: vpaddq %xmm0, %xmm0, %xmm0
-; AVX1-NEXT: vpsllq %xmm3, %xmm0, %xmm3
+; AVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm3
+; AVX1-NEXT: vpshufd {{.*#+}} xmm2 = xmm2[2,3,2,3]
; AVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm0
-; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm3[4,5,6,7]
+; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm3[0,1,2,3],xmm0[4,5,6,7]
; AVX1-NEXT: vpor %xmm1, %xmm0, %xmm0
; AVX1-NEXT: retq
;
@@ -158,13 +158,13 @@ define <2 x i64> @var_funnnel_v2i64(<2 x i64> %x, <2 x i64> %y, <2 x i64> %amt)
; XOPAVX1-LABEL: var_funnnel_v2i64:
; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vpmovsxbq {{.*#+}} xmm3 = [63,63]
-; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm4
-; XOPAVX1-NEXT: vpxor %xmm5, %xmm5, %xmm5
-; XOPAVX1-NEXT: vpsubq %xmm4, %xmm5, %xmm4
-; XOPAVX1-NEXT: vpshlq %xmm4, %xmm1, %xmm1
-; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm4
; XOPAVX1-NEXT: vpaddq %xmm0, %xmm0, %xmm0
-; XOPAVX1-NEXT: vpshlq %xmm2, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpshlq %xmm4, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
+; XOPAVX1-NEXT: vpsubq %xmm2, %xmm3, %xmm2
+; XOPAVX1-NEXT: vpshlq %xmm2, %xmm1, %xmm1
; XOPAVX1-NEXT: vpor %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: retq
;
@@ -366,13 +366,13 @@ define <4 x i32> @var_funnnel_v4i32(<4 x i32> %x, <4 x i32> %y, <4 x i32> %amt)
; XOPAVX1-LABEL: var_funnnel_v4i32:
; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vbroadcastss {{.*#+}} xmm3 = [31,31,31,31]
-; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm4
-; XOPAVX1-NEXT: vpxor %xmm5, %xmm5, %xmm5
-; XOPAVX1-NEXT: vpsubd %xmm4, %xmm5, %xmm4
-; XOPAVX1-NEXT: vpshld %xmm4, %xmm1, %xmm1
-; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm4
; XOPAVX1-NEXT: vpaddd %xmm0, %xmm0, %xmm0
-; XOPAVX1-NEXT: vpshld %xmm2, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpshld %xmm4, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
+; XOPAVX1-NEXT: vpsubd %xmm2, %xmm3, %xmm2
+; XOPAVX1-NEXT: vpshld %xmm2, %xmm1, %xmm1
; XOPAVX1-NEXT: vpor %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: retq
;
@@ -646,26 +646,26 @@ define <8 x i16> @var_funnnel_v8i16(<8 x i16> %x, <8 x i16> %y, <8 x i16> %amt)
; XOPAVX1-LABEL: var_funnnel_v8i16:
; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vbroadcastss {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15]
-; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm4
-; XOPAVX1-NEXT: vpxor %xmm5, %xmm5, %xmm5
-; XOPAVX1-NEXT: vpsubw %xmm4, %xmm5, %xmm4
-; XOPAVX1-NEXT: vpshlw %xmm4, %xmm1, %xmm1
-; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm4
; XOPAVX1-NEXT: vpaddw %xmm0, %xmm0, %xmm0
-; XOPAVX1-NEXT: vpshlw %xmm2, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpshlw %xmm4, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
+; XOPAVX1-NEXT: vpsubw %xmm2, %xmm3, %xmm2
+; XOPAVX1-NEXT: vpshlw %xmm2, %xmm1, %xmm1
; XOPAVX1-NEXT: vpor %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: var_funnnel_v8i16:
; XOPAVX2: # %bb.0:
; XOPAVX2-NEXT: vpbroadcastw {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15]
-; XOPAVX2-NEXT: vpand %xmm3, %xmm2, %xmm4
-; XOPAVX2-NEXT: vpxor %xmm5, %xmm5, %xmm5
-; XOPAVX2-NEXT: vpsubw %xmm4, %xmm5, %xmm4
-; XOPAVX2-NEXT: vpshlw %xmm4, %xmm1, %xmm1
-; XOPAVX2-NEXT: vpandn %xmm3, %xmm2, %xmm2
+; XOPAVX2-NEXT: vpandn %xmm3, %xmm2, %xmm4
; XOPAVX2-NEXT: vpaddw %xmm0, %xmm0, %xmm0
-; XOPAVX2-NEXT: vpshlw %xmm2, %xmm0, %xmm0
+; XOPAVX2-NEXT: vpshlw %xmm4, %xmm0, %xmm0
+; XOPAVX2-NEXT: vpand %xmm3, %xmm2, %xmm2
+; XOPAVX2-NEXT: vpxor %xmm3, %xmm3, %xmm3
+; XOPAVX2-NEXT: vpsubw %xmm2, %xmm3, %xmm2
+; XOPAVX2-NEXT: vpshlw %xmm2, %xmm1, %xmm1
; XOPAVX2-NEXT: vpor %xmm1, %xmm0, %xmm0
; XOPAVX2-NEXT: retq
;
@@ -995,26 +995,26 @@ define <16 x i8> @var_funnnel_v16i8(<16 x i8> %x, <16 x i8> %y, <16 x i8> %amt)
; XOPAVX1-LABEL: var_funnnel_v16i8:
; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vbroadcastss {{.*#+}} xmm3 = [7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7]
-; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm4
-; XOPAVX1-NEXT: vpxor %xmm5, %xmm5, %xmm5
-; XOPAVX1-NEXT: vpsubb %xmm4, %xmm5, %xmm4
-; XOPAVX1-NEXT: vpshlb %xmm4, %xmm1, %xmm1
-; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm4
; XOPAVX1-NEXT: vpaddb %xmm0, %xmm0, %xmm0
-; XOPAVX1-NEXT: vpshlb %xmm2, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpshlb %xmm4, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
+; XOPAVX1-NEXT: vpsubb %xmm2, %xmm3, %xmm2
+; XOPAVX1-NEXT: vpshlb %xmm2, %xmm1, %xmm1
; XOPAVX1-NEXT: vpor %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: var_funnnel_v16i8:
; XOPAVX2: # %bb.0:
; XOPAVX2-NEXT: vpbroadcastb {{.*#+}} xmm3 = [7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7]
-; XOPAVX2-NEXT: vpand %xmm3, %xmm2, %xmm4
-; XOPAVX2-NEXT: vpxor %xmm5, %xmm5, %xmm5
-; XOPAVX2-NEXT: vpsubb %xmm4, %xmm5, %xmm4
-; XOPAVX2-NEXT: vpshlb %xmm4, %xmm1, %xmm1
-; XOPAVX2-NEXT: vpandn %xmm3, %xmm2, %xmm2
+; XOPAVX2-NEXT: vpandn %xmm3, %xmm2, %xmm4
; XOPAVX2-NEXT: vpaddb %xmm0, %xmm0, %xmm0
-; XOPAVX2-NEXT: vpshlb %xmm2, %xmm0, %xmm0
+; XOPAVX2-NEXT: vpshlb %xmm4, %xmm0, %xmm0
+; XOPAVX2-NEXT: vpand %xmm3, %xmm2, %xmm2
+; XOPAVX2-NEXT: vpxor %xmm3, %xmm3, %xmm3
+; XOPAVX2-NEXT: vpsubb %xmm2, %xmm3, %xmm2
+; XOPAVX2-NEXT: vpshlb %xmm2, %xmm1, %xmm1
; XOPAVX2-NEXT: vpor %xmm1, %xmm0, %xmm0
; XOPAVX2-NEXT: retq
;
diff --git a/llvm/test/CodeGen/X86/vector-fshr-256.ll b/llvm/test/CodeGen/X86/vector-fshr-256.ll
index 3fabf720da71c3..659e6feb91d5c3 100644
--- a/llvm/test/CodeGen/X86/vector-fshr-256.ll
+++ b/llvm/test/CodeGen/X86/vector-fshr-256.ll
@@ -486,22 +486,22 @@ define <16 x i16> @var_funnnel_v16i16(<16 x i16> %x, <16 x i16> %y, <16 x i16> %
; XOPAVX2-LABEL: var_funnnel_v16i16:
; XOPAVX2: # %bb.0:
; XOPAVX2-NEXT: vpbroadcastw {{.*#+}} ymm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
-; XOPAVX2-NEXT: vpand %ymm3, %ymm2, %ymm4
+; XOPAVX2-NEXT: vpandn %ymm3, %ymm2, %ymm4
; XOPAVX2-NEXT: vextracti128 $1, %ymm4, %xmm5
-; XOPAVX2-NEXT: vpxor %xmm6, %xmm6, %xmm6
-; XOPAVX2-NEXT: vpsubw %xmm5, %xmm6, %xmm5
-; XOPAVX2-NEXT: vextracti128 $1, %ymm1, %xmm7
-; XOPAVX2-NEXT: vpshlw %xmm5, %xmm7, %xmm5
-; XOPAVX2-NEXT: vpsubw %xmm4, %xmm6, %xmm4
-; XOPAVX2-NEXT: vpshlw %xmm4, %xmm1, %xmm1
-; XOPAVX2-NEXT: vinserti128 $1, %xmm5, %ymm1, %ymm1
-; XOPAVX2-NEXT: vpandn %ymm3, %ymm2, %ymm2
-; XOPAVX2-NEXT: vextracti128 $1, %ymm2, %xmm3
; XOPAVX2-NEXT: vpaddw %ymm0, %ymm0, %ymm0
-; XOPAVX2-NEXT: vextracti128 $1, %ymm0, %xmm4
-; XOPAVX2-NEXT: vpshlw %xmm3, %xmm4, %xmm3
-; XOPAVX2-NEXT: vpshlw %xmm2, %xmm0, %xmm0
-; XOPAVX2-NEXT: vinserti128 $1, %xmm3, %ymm0, %ymm0
+; XOPAVX2-NEXT: vextracti128 $1, %ymm0, %xmm6
+; XOPAVX2-NEXT: vpshlw %xmm5, %xmm6, %xmm5
+; XOPAVX2-NEXT: vpshlw %xmm4, %xmm0, %xmm0
+; XOPAVX2-NEXT: vinserti128 $1, %xmm5, %ymm0, %ymm0
+; XOPAVX2-NEXT: vpand %ymm3, %ymm2, %ymm2
+; XOPAVX2-NEXT: vextracti128 $1, %ymm2, %xmm3
+; XOPAVX2-NEXT: vpxor %xmm4, %xmm4, %xmm4
+; XOPAVX2-NEXT: vpsubw %xmm3, %xmm4, %xmm3
+; XOPAVX2-NEXT: vextracti128 $1, %ymm1, %xmm5
+; XOPAVX2-NEXT: vpshlw %xmm3, %xmm5, %xmm3
+; XOPAVX2-NEXT: vpsubw %xmm2, %xmm4, %xmm2
+; XOPAVX2-NEXT: vpshlw %xmm2, %xmm1, %xmm1
+; XOPAVX2-NEXT: vinserti128 $1, %xmm3, %ymm1, %ymm1
; XOPAVX2-NEXT: vpor %ymm1, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
%res = call <16 x i16> @llvm.fshr.v16i16(<16 x i16> %x, <16 x i16> %y, <16 x i16> %amt)
diff --git a/llvm/test/CodeGen/X86/vector-fshr-sub128.ll b/llvm/test/CodeGen/X86/vector-fshr-sub128.ll
index 0426c48aecfcff..a6067a960fc0d6 100644
--- a/llvm/test/CodeGen/X86/vector-fshr-sub128.ll
+++ b/llvm/test/CodeGen/X86/vector-fshr-sub128.ll
@@ -185,13 +185,13 @@ define <2 x i32> @var_funnnel_v2i32(<2 x i32> %x, <2 x i32> %y, <2 x i32> %amt)
; XOPAVX1-LABEL: var_funnnel_v2i32:
; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vbroadcastss {{.*#+}} xmm3 = [31,31,31,31]
-; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm4
-; XOPAVX1-NEXT: vpxor %xmm5, %xmm5, %xmm5
-; XOPAVX1-NEXT: vpsubd %xmm4, %xmm5, %xmm4
-; XOPAVX1-NEXT: vpshld %xmm4, %xmm1, %xmm1
-; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpandn %xmm3, %xmm2, %xmm4
; XOPAVX1-NEXT: vpaddd %xmm0, %xmm0, %xmm0
-; XOPAVX1-NEXT: vpshld %xmm2, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpshld %xmm4, %xmm0, %xmm0
+; XOPAVX1-NEXT: vpand %xmm3, %xmm2, %xmm2
+; XOPAVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
+; XOPAVX1-NEXT: vpsubd %xmm2, %xmm3, %xmm2
+; XOPAVX1-NEXT: vpshld %xmm2, %xmm1, %xmm1
; XOPAVX1-NEXT: vpor %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: retq
;
diff --git a/llvm/test/CodeGen/X86/vector-shift-shl-128.ll b/llvm/test/CodeGen/X86/vector-shift-shl-128.ll
index c54da38ef10cc1..75baba5f35f792 100644
--- a/llvm/test/CodeGen/X86/vector-shift-shl-128.ll
+++ b/llvm/test/CodeGen/X86/vector-shift-shl-128.ll
@@ -927,9 +927,9 @@ define <2 x i64> @constant_shift_v2i64(<2 x i64> %a) nounwind {
; SSE2-LABEL: constant_shift_v2i64:
; SSE2: # %bb.0:
; SSE2-NEXT: movdqa %xmm0, %xmm1
-; SSE2-NEXT: psllq $7, %xmm1
-; SSE2-NEXT: paddq %xmm0, %xmm0
-; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[2,3]
+; SSE2-NEXT: paddq %xmm0, %xmm1
+; SSE2-NEXT: psllq $7, %xmm0
+; SSE2-NEXT: movsd {{.*#+}} xmm0 = xmm1[0],xmm0[1]
; SSE2-NEXT: retq
;
; SSE41-LABEL: constant_shift_v2i64:
@@ -975,9 +975,9 @@ define <2 x i64> @constant_shift_v2i64(<2 x i64> %a) nounwind {
; X86-SSE-LABEL: constant_shift_v2i64:
; X86-SSE: # %bb.0:
; X86-SSE-NEXT: movdqa %xmm0, %xmm1
-; X86-SSE-NEXT: psllq $7, %xmm1
-; X86-SSE-NEXT: paddq %xmm0, %xmm0
-; X86-SSE-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[2,3]
+; X86-SSE-NEXT: paddq %xmm0, %xmm1
+; X86-SSE-NEXT: psllq $7, %xmm0
+; X86-SSE-NEXT: movsd {{.*#+}} xmm0 = xmm1[0],xmm0[1]
; X86-SSE-NEXT: retl
%shift = shl <2 x i64> %a, <i64 1, i64 7>
ret <2 x i64> %shift
More information about the llvm-commits
mailing list