[llvm] RegisterPressure: Remove dead defs correctly (PR #222627)
Matt Arsenault via llvm-commits
llvm-commits at lists.llvm.org
Thu Sep 10 05:32:09 PDT 2026
https://github.com/arsenm created https://github.com/llvm/llvm-project/pull/222627
When an instruction has overlapping register defs where only some carry the
dead flag (for example a dead super-register def alongside a live sub-register
def), the collector left the shared register units in both the live and dead def
sets. That produced a PressureDiff decrement with no matching increment and tripped
the "PSet overflow/underflow" assertion in getUpwardPressureDelta.
A register unit is dead if any def covering it is dead, regardless of operand
order. Reconcile by subtracting the dead defs from the live defs instead of the
reverse: removeRegLanes only clears the overlapping units, so a def keeps any
units a dead def does not cover.
Fixes #155807.
Fixes #149144.
Fixes #76416.
Fixes #205272.
Co-authored-by: XChy <xxs_chy at outlook.com>
Co-authored-by: Claude claude-opus-4.8 <noreply at anthropic.com>
>From 30f6f8641c48d53abb78bcd5f03bdb74d5b6ff47 Mon Sep 17 00:00:00 2001
From: Matt Arsenault <Matthew.Arsenault at amd.com>
Date: Thu, 10 Sep 2026 13:04:03 +0200
Subject: [PATCH] RegisterPressure: Remove dead defs correctly
When an instruction has overlapping register defs where only some carry the
dead flag (for example a dead super-register def alongside a live sub-register
def), the collector left the shared register units in both the live and dead def
sets. That produced a PressureDiff decrement with no matching increment and tripped
the "PSet overflow/underflow" assertion in getUpwardPressureDelta.
A register unit is dead if any def covering it is dead, regardless of operand
order. Reconcile by subtracting the dead defs from the live defs instead of the
reverse: removeRegLanes only clears the overlapping units, so a def keeps any
units a dead def does not cover.
Fixes #155807.
Fixes #149144.
Fixes #76416.
Fixes #205272.
Co-authored-by: XChy <xxs_chy at outlook.com>
Co-authored-by: Claude claude-opus-4.8 <noreply at anthropic.com>
---
llvm/lib/CodeGen/RegisterPressure.cpp | 19 +-
.../CodeGen/X86/dead-register-pr76416.mir | 35 ++
llvm/test/CodeGen/X86/inline-asm-pr149144.ll | 26 ++
llvm/test/CodeGen/X86/inline-asm-pr155807.ll | 21 +
llvm/test/CodeGen/X86/inline-asm-pr76416.ll | 75 +++
llvm/test/CodeGen/X86/masked-udiv.ll | 433 +++++++++---------
.../CodeGen/X86/min-legal-vector-width.ll | 64 +--
llvm/test/CodeGen/X86/pr205272.ll | 23 +
llvm/test/CodeGen/X86/ssub_sat_plus.ll | 4 +-
llvm/test/CodeGen/X86/udiv_fix.ll | 34 +-
llvm/test/CodeGen/X86/udiv_fix_sat.ll | 82 ++--
llvm/test/CodeGen/X86/usub_sat_plus.ll | 4 +-
llvm/test/CodeGen/X86/vector-idiv-strictfp.ll | 191 ++++----
llvm/test/CodeGen/X86/vector-idiv-udiv-512.ll | 132 +++---
llvm/test/CodeGen/X86/xmulo.ll | 12 +-
15 files changed, 696 insertions(+), 459 deletions(-)
create mode 100644 llvm/test/CodeGen/X86/dead-register-pr76416.mir
create mode 100644 llvm/test/CodeGen/X86/inline-asm-pr149144.ll
create mode 100644 llvm/test/CodeGen/X86/inline-asm-pr155807.ll
create mode 100644 llvm/test/CodeGen/X86/inline-asm-pr76416.ll
create mode 100644 llvm/test/CodeGen/X86/pr205272.ll
diff --git a/llvm/lib/CodeGen/RegisterPressure.cpp b/llvm/lib/CodeGen/RegisterPressure.cpp
index c42dced2a5bea..5956f52764c90 100644
--- a/llvm/lib/CodeGen/RegisterPressure.cpp
+++ b/llvm/lib/CodeGen/RegisterPressure.cpp
@@ -472,18 +472,25 @@ class RegisterOperandsCollector {
for (ConstMIBundleOperands OperI(MI); OperI.isValid(); ++OperI)
collectOperand(*OperI);
- // Remove redundant physreg dead defs.
- for (const VRegMaskOrUnit &P : RegOpers.Defs)
- removeRegLanes(RegOpers.DeadDefs, P);
+ // An instruction can have overlapping defs where only some carry the dead
+ // flag, for example a dead super-register def alongside a live
+ // sub-register def. A register unit is dead if any def covering it is dead,
+ // so subtract the dead defs from the live defs. removeRegLanes only clears
+ // the overlapping units, leaving each def with the units no dead def
+ // covers.
+ for (const VRegMaskOrUnit &P : RegOpers.DeadDefs)
+ removeRegLanes(RegOpers.Defs, P);
}
void collectInstrLanes(const MachineInstr &MI) const {
for (ConstMIBundleOperands OperI(MI); OperI.isValid(); ++OperI)
collectOperandLanes(*OperI);
- // Remove redundant physreg dead defs.
- for (const VRegMaskOrUnit &P : RegOpers.Defs)
- removeRegLanes(RegOpers.DeadDefs, P);
+ // A register unit is dead if any def covering it is dead; subtract the
+ // dead defs from the live defs so overlapping defs do not leave a unit
+ // counted as live. See collectInstr.
+ for (const VRegMaskOrUnit &P : RegOpers.DeadDefs)
+ removeRegLanes(RegOpers.Defs, P);
}
/// Push this operand's register onto the correct vectors.
diff --git a/llvm/test/CodeGen/X86/dead-register-pr76416.mir b/llvm/test/CodeGen/X86/dead-register-pr76416.mir
new file mode 100644
index 0000000000000..718b0545df436
--- /dev/null
+++ b/llvm/test/CodeGen/X86/dead-register-pr76416.mir
@@ -0,0 +1,35 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=x86_64-unknown-unknown -run-pass=machine-scheduler -verify-misched -o - %s | FileCheck %s
+
+# An overlapping dead super-register def (implicit-def $rax) and dead
+# sub-register def ($eax) must not leave a phantom live register unit, which
+# previously unbalanced the register pressure tracker.
+
+---
+name: vga_load_state
+tracksRegLiveness: true
+body: |
+ bb.0:
+ liveins: $rdi
+
+ ; CHECK-LABEL: name: vga_load_state
+ ; CHECK: liveins: $rdi
+ ; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: dead [[COPY:%[0-9]+]]:gr64 = COPY $rdi
+ ; CHECK-NEXT: dead undef [[MOV32r0_:%[0-9]+]].sub_32bit:gr64_nosp = MOV32r0 implicit-def dead $eflags
+ ; CHECK-NEXT: INLINEASM &"", sideeffect attdialect
+ ; CHECK-NEXT: INLINEASM &"", sideeffect attdialect
+ ; CHECK-NEXT: dead $eax = MOV32r0 implicit-def dead $eflags, implicit-def $rax
+ ; CHECK-NEXT: dead $eax = MOV32r0 implicit-def dead $eflags
+ ; CHECK-NEXT: dead $eax = MOV32r0 implicit-def dead $eflags
+ ; CHECK-NEXT: dead $eax = MOV32r0 implicit-def dead $eflags, implicit-def $rax
+ %0:gr64 = COPY $rdi
+ undef %1.sub_32bit:gr64_nosp = MOV32r0 implicit-def dead $eflags
+ dead $eax = MOV32r0 implicit-def dead $eflags, implicit-def $rax
+ INLINEASM &"", 1 /* sideeffect attdialect */
+ dead $eax = MOV32r0 implicit-def dead $eflags
+ INLINEASM &"", 1 /* sideeffect attdialect */
+ dead $eax = MOV32r0 implicit-def dead $eflags
+ dead $eax = MOV32r0 implicit-def dead $eflags, implicit-def $rax
+
+...
diff --git a/llvm/test/CodeGen/X86/inline-asm-pr149144.ll b/llvm/test/CodeGen/X86/inline-asm-pr149144.ll
new file mode 100644
index 0000000000000..422412246c1f2
--- /dev/null
+++ b/llvm/test/CodeGen/X86/inline-asm-pr149144.ll
@@ -0,0 +1,26 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -mtriple=x86_64 < %s | FileCheck %s
+
+; An inline asm with an "=a" output tied to an input leaves the scheduler with
+; overlapping dead and live physreg defs. The register pressure tracker must
+; not count the shared units as live, or it hits a "PSet overflow/underflow"
+; assertion.
+define i32 @pr149144(i8 %how, ptr %set, ptr %oset) {
+; CHECK-LABEL: pr149144:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: pushq %rbx
+; CHECK-NEXT: .cfi_def_cfa_offset 16
+; CHECK-NEXT: .cfi_offset %rbx, -16
+; CHECK-NEXT: movq %rsi, %rcx
+; CHECK-NEXT: movl $14, %ebx
+; CHECK-NEXT: movb %dil, %bl
+; CHECK-NEXT: movl $8, %esi
+; CHECK-NEXT: #APP
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: popq %rbx
+; CHECK-NEXT: .cfi_def_cfa_offset 8
+; CHECK-NEXT: retq
+entry:
+ %0 = tail call i32 asm sideeffect "", "={ax},0{bx}0,{bx},{cx},{dx},{si},~{memory},~{cc},~{dirflag},~{fpsr},~{flags}"(i32 14, i8 %how, ptr %set, ptr %oset, i32 8)
+ ret i32 %0
+}
diff --git a/llvm/test/CodeGen/X86/inline-asm-pr155807.ll b/llvm/test/CodeGen/X86/inline-asm-pr155807.ll
new file mode 100644
index 0000000000000..3c10773dbb66c
--- /dev/null
+++ b/llvm/test/CodeGen/X86/inline-asm-pr155807.ll
@@ -0,0 +1,21 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -mtriple=x86_64 < %s | FileCheck %s
+
+; LiveVariable Analysis transforms (inlineasm rax, eax) to (inlineasm dead rax, eax)
+; instead of (inlineasm dead rax, dead eax). RegisterPressure fails at considering
+; eax as dead register.
+define i64 @pr155807(i64 %vecext) {
+; CHECK-LABEL: pr155807:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: #APP
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: movabsq $705425148255374, %rax # imm = 0x2819497609C8E
+; CHECK-NEXT: xorl %edx, %edx
+; CHECK-NEXT: idivq %rdi
+; CHECK-NEXT: movq %rdx, %rax
+; CHECK-NEXT: retq
+entry:
+ tail call void asm sideeffect "", "~{rax},~{eax},~{dirflag},~{fpsr},~{flags}"()
+ %rem = srem i64 705425148255374, %vecext
+ ret i64 %rem
+}
diff --git a/llvm/test/CodeGen/X86/inline-asm-pr76416.ll b/llvm/test/CodeGen/X86/inline-asm-pr76416.ll
new file mode 100644
index 0000000000000..b14a6eda7a6a6
--- /dev/null
+++ b/llvm/test/CodeGen/X86/inline-asm-pr76416.ll
@@ -0,0 +1,75 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -O2 -mtriple=x86_64 < %s | FileCheck %s
+%struct.anon.5.28.78.99.149.119 = type { [4 x i8] }
+
+ at vga_load_state_p = external dso_local global ptr, align 8
+ at vga_load_state_data = external dso_local global i8, align 1
+
+define dso_local void @vga_load_state() #0 {
+; CHECK-LABEL: vga_load_state:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: movl $0, -{{[0-9]+}}(%rsp)
+; CHECK-NEXT: cmpl $3, -{{[0-9]+}}(%rsp)
+; CHECK-NEXT: jg .LBB0_3
+; CHECK-NEXT: .p2align 4
+; CHECK-NEXT: .LBB0_2: # %for.body
+; CHECK-NEXT: # =>This Inner Loop Header: Depth=1
+; CHECK-NEXT: xorl %eax, %eax
+; CHECK-NEXT: #APP
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: incl -{{[0-9]+}}(%rsp)
+; CHECK-NEXT: cmpl $3, -{{[0-9]+}}(%rsp)
+; CHECK-NEXT: jle .LBB0_2
+; CHECK-NEXT: .LBB0_3: # %for.end
+; CHECK-NEXT: movl $0, -{{[0-9]+}}(%rsp)
+; CHECK-NEXT: .p2align 4
+; CHECK-NEXT: .LBB0_4: # %for.cond1
+; CHECK-NEXT: # =>This Inner Loop Header: Depth=1
+; CHECK-NEXT: # implicit-def: $edx
+; CHECK-NEXT: #APP
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: movq vga_load_state_p(%rip), %rax
+; CHECK-NEXT: movslq -{{[0-9]+}}(%rsp), %rcx
+; CHECK-NEXT: movzbl (%rax,%rcx), %eax
+; CHECK-NEXT: movb %al, vga_load_state_data(%rip)
+; CHECK-NEXT: leal 1(%rcx), %eax
+; CHECK-NEXT: movl %eax, -{{[0-9]+}}(%rsp)
+; CHECK-NEXT: jmp .LBB0_4
+entry:
+ %i = alloca i32, align 4
+ store i32 0, ptr %i, align 4
+ br label %for.cond
+
+for.cond: ; preds = %for.body, %entry
+ %0 = load i32, ptr %i, align 4
+ %cmp = icmp slt i32 %0, 4
+ br i1 %cmp, label %for.body, label %for.end
+
+for.body: ; preds = %for.cond
+ call void asm sideeffect "", "{ax},~{dirflag},~{fpsr},~{flags}"(i8 0) #2
+ %1 = load i32, ptr %i, align 4
+ %inc = add nsw i32 %1, 1
+ store i32 %inc, ptr %i, align 4
+ br label %for.cond
+
+for.end: ; preds = %for.cond
+ store i32 0, ptr %i, align 4
+ br label %for.cond1
+
+for.cond1: ; preds = %for.cond1, %for.end
+ call void asm sideeffect "", "N{dx},~{dirflag},~{fpsr},~{flags}"(i32 poison) #2
+ %2 = load ptr, ptr @vga_load_state_p, align 8
+ %regs = getelementptr inbounds %struct.anon.5.28.78.99.149.119, ptr %2, i32 0, i32 0
+ %3 = load i32, ptr %i, align 4
+ %idxprom = sext i32 %3 to i64
+ %arrayidx = getelementptr inbounds [4 x i8], ptr %regs, i64 0, i64 %idxprom
+ %4 = load i8, ptr %arrayidx, align 1
+ store i8 %4, ptr @vga_load_state_data, align 1
+ %5 = load i32, ptr %i, align 4
+ %inc5 = add nsw i32 %5, 1
+ store i32 %inc5, ptr %i, align 4
+ br label %for.cond1, !llvm.loop !1
+}
+
+!1 = distinct !{!1, !2}
+!2 = !{!"llvm.loop.mustprogress"}
diff --git a/llvm/test/CodeGen/X86/masked-udiv.ll b/llvm/test/CodeGen/X86/masked-udiv.ll
index 2903265589a0b..28c16a1447ec5 100644
--- a/llvm/test/CodeGen/X86/masked-udiv.ll
+++ b/llvm/test/CodeGen/X86/masked-udiv.ll
@@ -204,39 +204,40 @@ define <2 x i64> @udiv_v2i64(<2 x i64> %x, <2 x i64> %y, <2 x i1> %m) {
define <4 x i64> @udiv_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i1> %m) {
; SSE2-LABEL: udiv_v4i64:
; SSE2: # %bb.0:
-; SSE2-NEXT: movdqa %xmm0, %xmm5
-; SSE2-NEXT: pshufd {{.*#+}} xmm6 = xmm4[2,2,3,3]
+; SSE2-NEXT: pshufd {{.*#+}} xmm6 = xmm4[0,0,1,1]
; SSE2-NEXT: pslld $31, %xmm6
; SSE2-NEXT: psrad $31, %xmm6
-; SSE2-NEXT: pshufd {{.*#+}} xmm4 = xmm4[0,0,1,1]
-; SSE2-NEXT: pslld $31, %xmm4
-; SSE2-NEXT: psrad $31, %xmm4
-; SSE2-NEXT: movdqa {{.*#+}} xmm7 = [1,1]
-; SSE2-NEXT: pand %xmm4, %xmm2
-; SSE2-NEXT: pandn %xmm7, %xmm4
-; SSE2-NEXT: por %xmm2, %xmm4
-; SSE2-NEXT: movq %xmm4, %rcx
+; SSE2-NEXT: movdqa {{.*#+}} xmm5 = [1,1]
+; SSE2-NEXT: pand %xmm6, %xmm2
+; SSE2-NEXT: pandn %xmm5, %xmm6
+; SSE2-NEXT: por %xmm2, %xmm6
+; SSE2-NEXT: movq %xmm6, %rcx
; SSE2-NEXT: movq %xmm0, %rax
; SSE2-NEXT: xorl %edx, %edx
; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm0
-; SSE2-NEXT: pshufd {{.*#+}} xmm2 = xmm4[2,3,2,3]
-; SSE2-NEXT: movq %xmm2, %rcx
-; SSE2-NEXT: pshufd {{.*#+}} xmm2 = xmm5[2,3,2,3]
-; SSE2-NEXT: movq %xmm2, %rax
+; SSE2-NEXT: movq %rax, %rcx
+; SSE2-NEXT: pshufd {{.*#+}} xmm2 = xmm6[2,3,2,3]
+; SSE2-NEXT: movq %xmm2, %rsi
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
+; SSE2-NEXT: movq %xmm0, %rax
; SSE2-NEXT: xorl %edx, %edx
-; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm2
-; SSE2-NEXT: punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm2[0]
-; SSE2-NEXT: pand %xmm6, %xmm3
-; SSE2-NEXT: pandn %xmm7, %xmm6
-; SSE2-NEXT: por %xmm3, %xmm6
-; SSE2-NEXT: movq %xmm6, %rcx
+; SSE2-NEXT: divq %rsi
+; SSE2-NEXT: movq %rax, %rsi
+; SSE2-NEXT: pshufd {{.*#+}} xmm4 = xmm4[2,2,3,3]
+; SSE2-NEXT: pslld $31, %xmm4
+; SSE2-NEXT: psrad $31, %xmm4
+; SSE2-NEXT: pand %xmm4, %xmm3
+; SSE2-NEXT: pandn %xmm5, %xmm4
+; SSE2-NEXT: por %xmm3, %xmm4
+; SSE2-NEXT: movq %xmm4, %rdi
; SSE2-NEXT: movq %xmm1, %rax
; SSE2-NEXT: xorl %edx, %edx
-; SSE2-NEXT: divq %rcx
+; SSE2-NEXT: divq %rdi
+; SSE2-NEXT: movq %rcx, %xmm0
+; SSE2-NEXT: movq %rsi, %xmm2
+; SSE2-NEXT: punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm2[0]
; SSE2-NEXT: movq %rax, %xmm2
-; SSE2-NEXT: pshufd {{.*#+}} xmm3 = xmm6[2,3,2,3]
+; SSE2-NEXT: pshufd {{.*#+}} xmm3 = xmm4[2,3,2,3]
; SSE2-NEXT: movq %xmm3, %rcx
; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm1[2,3,2,3]
; SSE2-NEXT: movq %xmm1, %rax
@@ -250,38 +251,39 @@ define <4 x i64> @udiv_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i1> %m) {
; SSE42-LABEL: udiv_v4i64:
; SSE42: # %bb.0:
; SSE42-NEXT: movdqa %xmm0, %xmm5
-; SSE42-NEXT: pshufd {{.*#+}} xmm6 = xmm4[2,2,3,3]
; SSE42-NEXT: pmovzxdq {{.*#+}} xmm0 = xmm4[0],zero,xmm4[1],zero
; SSE42-NEXT: psllq $63, %xmm0
-; SSE42-NEXT: movapd {{.*#+}} xmm4 = [1,1]
-; SSE42-NEXT: movapd %xmm4, %xmm7
+; SSE42-NEXT: movapd {{.*#+}} xmm6 = [1,1]
+; SSE42-NEXT: movapd %xmm6, %xmm7
; SSE42-NEXT: blendvpd %xmm0, %xmm2, %xmm7
; SSE42-NEXT: pextrq $1, %xmm7, %rcx
; SSE42-NEXT: pextrq $1, %xmm5, %rax
-; SSE42-NEXT: psllq $63, %xmm6
; SSE42-NEXT: xorl %edx, %edx
; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm8
-; SSE42-NEXT: movq %xmm7, %rcx
+; SSE42-NEXT: movq %rax, %rcx
+; SSE42-NEXT: pshufd {{.*#+}} xmm0 = xmm4[2,2,3,3]
+; SSE42-NEXT: psllq $63, %xmm0
+; SSE42-NEXT: movq %xmm7, %rdi
+; SSE42-NEXT: blendvpd %xmm0, %xmm3, %xmm6
+; SSE42-NEXT: pextrq $1, %xmm6, %r8
+; SSE42-NEXT: pextrq $1, %xmm1, %rsi
; SSE42-NEXT: movq %xmm5, %rax
; SSE42-NEXT: xorl %edx, %edx
-; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm2
-; SSE42-NEXT: movdqa %xmm6, %xmm0
-; SSE42-NEXT: blendvpd %xmm0, %xmm3, %xmm4
-; SSE42-NEXT: pextrq $1, %xmm4, %rcx
-; SSE42-NEXT: pextrq $1, %xmm1, %rax
-; SSE42-NEXT: punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm8[0]
+; SSE42-NEXT: divq %rdi
+; SSE42-NEXT: movq %rax, %rdi
+; SSE42-NEXT: movq %rsi, %rax
; SSE42-NEXT: xorl %edx, %edx
-; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm0
-; SSE42-NEXT: movq %xmm4, %rcx
+; SSE42-NEXT: divq %r8
+; SSE42-NEXT: movq %rcx, %xmm2
+; SSE42-NEXT: movq %rdi, %xmm0
+; SSE42-NEXT: punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm2[0]
+; SSE42-NEXT: movq %rax, %xmm2
+; SSE42-NEXT: movq %xmm6, %rcx
; SSE42-NEXT: movq %xmm1, %rax
; SSE42-NEXT: xorl %edx, %edx
; SSE42-NEXT: divq %rcx
; SSE42-NEXT: movq %rax, %xmm1
-; SSE42-NEXT: punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]
-; SSE42-NEXT: movdqa %xmm2, %xmm0
+; SSE42-NEXT: punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm2[0]
; SSE42-NEXT: retq
;
; AVX2-LABEL: udiv_v4i64:
@@ -301,19 +303,20 @@ define <4 x i64> @udiv_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i1> %m) {
; AVX2-NEXT: vmovq %xmm3, %rax
; AVX2-NEXT: xorl %edx, %edx
; AVX2-NEXT: divq %rsi
-; AVX2-NEXT: vmovq %rcx, %xmm2
-; AVX2-NEXT: vpextrq $1, %xmm1, %rcx
-; AVX2-NEXT: vmovq %rax, %xmm3
+; AVX2-NEXT: movq %rax, %rsi
+; AVX2-NEXT: vpextrq $1, %xmm1, %rdi
; AVX2-NEXT: vpextrq $1, %xmm0, %rax
; AVX2-NEXT: xorl %edx, %edx
-; AVX2-NEXT: divq %rcx
-; AVX2-NEXT: movq %rax, %rcx
+; AVX2-NEXT: divq %rdi
+; AVX2-NEXT: movq %rax, %rdi
+; AVX2-NEXT: vmovq %rcx, %xmm2
+; AVX2-NEXT: vmovq %rsi, %xmm3
; AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm3[0],xmm2[0]
-; AVX2-NEXT: vmovq %xmm1, %rsi
+; AVX2-NEXT: vmovq %xmm1, %rcx
; AVX2-NEXT: vmovq %xmm0, %rax
; AVX2-NEXT: xorl %edx, %edx
-; AVX2-NEXT: divq %rsi
-; AVX2-NEXT: vmovq %rcx, %xmm0
+; AVX2-NEXT: divq %rcx
+; AVX2-NEXT: vmovq %rdi, %xmm0
; AVX2-NEXT: vmovq %rax, %xmm1
; AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm0 = xmm1[0],xmm0[0]
; AVX2-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
@@ -816,169 +819,181 @@ define <3 x i10> @udiv_v3i10(<3 x i10> %x, <3 x i10> %y, <3 x i1> %m) {
define <8 x i64> @udiv_v8i64(<8 x i64> %x, <8 x i64> %y, <8 x i1> %m) {
; SSE2-LABEL: udiv_v8i64:
; SSE2: # %bb.0:
-; SSE2-NEXT: movdqa %xmm0, %xmm8
-; SSE2-NEXT: movdqa {{[0-9]+}}(%rsp), %xmm0
-; SSE2-NEXT: pshufd {{.*#+}} xmm9 = xmm0[3,3,3,3]
+; SSE2-NEXT: movdqa {{[0-9]+}}(%rsp), %xmm8
+; SSE2-NEXT: pshufd {{.*#+}} xmm9 = xmm8[0,0,0,0]
; SSE2-NEXT: pshufhw {{.*#+}} xmm9 = xmm9[0,1,2,3,5,5,5,5]
-; SSE2-NEXT: pshufd {{.*#+}} xmm9 = xmm9[0,0,2,2]
-; SSE2-NEXT: pslld $31, %xmm9
-; SSE2-NEXT: psrad $31, %xmm9
-; SSE2-NEXT: pshufd {{.*#+}} xmm10 = xmm0[2,2,2,2]
-; SSE2-NEXT: pshufhw {{.*#+}} xmm10 = xmm10[0,1,2,3,5,5,5,5]
-; SSE2-NEXT: pshufd {{.*#+}} xmm10 = xmm10[0,0,2,2]
+; SSE2-NEXT: pshufd {{.*#+}} xmm10 = xmm9[0,0,2,2]
; SSE2-NEXT: pslld $31, %xmm10
; SSE2-NEXT: psrad $31, %xmm10
-; SSE2-NEXT: pshufd {{.*#+}} xmm11 = xmm0[1,1,1,1]
-; SSE2-NEXT: pshufhw {{.*#+}} xmm11 = xmm11[0,1,2,3,5,5,5,5]
-; SSE2-NEXT: pshufd {{.*#+}} xmm12 = xmm11[0,0,2,2]
-; SSE2-NEXT: pslld $31, %xmm12
-; SSE2-NEXT: psrad $31, %xmm12
-; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm0[0,0,0,0]
-; SSE2-NEXT: pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,5,5,5,5]
-; SSE2-NEXT: pshufd {{.*#+}} xmm13 = xmm0[0,0,2,2]
-; SSE2-NEXT: pslld $31, %xmm13
-; SSE2-NEXT: psrad $31, %xmm13
-; SSE2-NEXT: movdqa {{.*#+}} xmm11 = [1,1]
-; SSE2-NEXT: pand %xmm13, %xmm4
-; SSE2-NEXT: pandn %xmm11, %xmm13
-; SSE2-NEXT: por %xmm4, %xmm13
-; SSE2-NEXT: movq %xmm13, %rcx
-; SSE2-NEXT: movq %xmm8, %rax
+; SSE2-NEXT: movdqa {{.*#+}} xmm9 = [1,1]
+; SSE2-NEXT: pand %xmm10, %xmm4
+; SSE2-NEXT: pandn %xmm9, %xmm10
+; SSE2-NEXT: por %xmm4, %xmm10
+; SSE2-NEXT: movq %xmm10, %rcx
+; SSE2-NEXT: movq %xmm0, %rax
; SSE2-NEXT: xorl %edx, %edx
; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm0
-; SSE2-NEXT: pshufd {{.*#+}} xmm4 = xmm13[2,3,2,3]
-; SSE2-NEXT: movq %xmm4, %rcx
-; SSE2-NEXT: pshufd {{.*#+}} xmm4 = xmm8[2,3,2,3]
-; SSE2-NEXT: movq %xmm4, %rax
+; SSE2-NEXT: movq %rax, %rcx
+; SSE2-NEXT: pshufd {{.*#+}} xmm4 = xmm10[2,3,2,3]
+; SSE2-NEXT: movq %xmm4, %rsi
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
+; SSE2-NEXT: movq %xmm0, %rax
; SSE2-NEXT: xorl %edx, %edx
-; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm4
-; SSE2-NEXT: punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm4[0]
-; SSE2-NEXT: pand %xmm12, %xmm5
-; SSE2-NEXT: pandn %xmm11, %xmm12
-; SSE2-NEXT: por %xmm5, %xmm12
-; SSE2-NEXT: movq %xmm12, %rcx
+; SSE2-NEXT: divq %rsi
+; SSE2-NEXT: movq %rax, %rsi
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm8[1,1,1,1]
+; SSE2-NEXT: pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,5,5,5,5]
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm0[0,0,2,2]
+; SSE2-NEXT: pslld $31, %xmm0
+; SSE2-NEXT: psrad $31, %xmm0
+; SSE2-NEXT: pand %xmm0, %xmm5
+; SSE2-NEXT: pandn %xmm9, %xmm0
+; SSE2-NEXT: por %xmm5, %xmm0
+; SSE2-NEXT: movq %xmm0, %rdi
; SSE2-NEXT: movq %xmm1, %rax
; SSE2-NEXT: xorl %edx, %edx
-; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm4
-; SSE2-NEXT: pshufd {{.*#+}} xmm5 = xmm12[2,3,2,3]
-; SSE2-NEXT: movq %xmm5, %rcx
-; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm1[2,3,2,3]
-; SSE2-NEXT: movq %xmm1, %rax
+; SSE2-NEXT: divq %rdi
+; SSE2-NEXT: movq %rax, %rdi
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
+; SSE2-NEXT: movq %xmm0, %r8
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm1[2,3,2,3]
+; SSE2-NEXT: movq %xmm0, %rax
; SSE2-NEXT: xorl %edx, %edx
-; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm1
-; SSE2-NEXT: punpcklqdq {{.*#+}} xmm4 = xmm4[0],xmm1[0]
-; SSE2-NEXT: pand %xmm10, %xmm6
-; SSE2-NEXT: pandn %xmm11, %xmm10
-; SSE2-NEXT: por %xmm6, %xmm10
-; SSE2-NEXT: movq %xmm10, %rcx
+; SSE2-NEXT: divq %r8
+; SSE2-NEXT: movq %rax, %r8
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm8[2,2,2,2]
+; SSE2-NEXT: pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,5,5,5,5]
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm0[0,0,2,2]
+; SSE2-NEXT: pslld $31, %xmm0
+; SSE2-NEXT: psrad $31, %xmm0
+; SSE2-NEXT: pand %xmm0, %xmm6
+; SSE2-NEXT: pandn %xmm9, %xmm0
+; SSE2-NEXT: por %xmm6, %xmm0
+; SSE2-NEXT: movq %xmm0, %r9
; SSE2-NEXT: movq %xmm2, %rax
; SSE2-NEXT: xorl %edx, %edx
-; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm5
-; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm10[2,3,2,3]
-; SSE2-NEXT: movq %xmm1, %rcx
-; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm2[2,3,2,3]
-; SSE2-NEXT: movq %xmm1, %rax
+; SSE2-NEXT: divq %r9
+; SSE2-NEXT: movq %rax, %r9
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
+; SSE2-NEXT: movq %xmm0, %r10
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm2[2,3,2,3]
+; SSE2-NEXT: movq %xmm0, %rax
; SSE2-NEXT: xorl %edx, %edx
-; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm1
-; SSE2-NEXT: punpcklqdq {{.*#+}} xmm5 = xmm5[0],xmm1[0]
-; SSE2-NEXT: pand %xmm9, %xmm7
-; SSE2-NEXT: pandn %xmm11, %xmm9
-; SSE2-NEXT: por %xmm7, %xmm9
-; SSE2-NEXT: movq %xmm9, %rcx
+; SSE2-NEXT: divq %r10
+; SSE2-NEXT: movq %rax, %r10
+; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm8[3,3,3,3]
+; SSE2-NEXT: pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,5,5,5,5]
+; SSE2-NEXT: pshufd {{.*#+}} xmm5 = xmm0[0,0,2,2]
+; SSE2-NEXT: pslld $31, %xmm5
+; SSE2-NEXT: psrad $31, %xmm5
+; SSE2-NEXT: pand %xmm5, %xmm7
+; SSE2-NEXT: pandn %xmm9, %xmm5
+; SSE2-NEXT: por %xmm7, %xmm5
+; SSE2-NEXT: movq %xmm5, %r11
; SSE2-NEXT: movq %xmm3, %rax
; SSE2-NEXT: xorl %edx, %edx
-; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm6
-; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm9[2,3,2,3]
-; SSE2-NEXT: movq %xmm1, %rcx
-; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm3[2,3,2,3]
-; SSE2-NEXT: movq %xmm1, %rax
+; SSE2-NEXT: divq %r11
+; SSE2-NEXT: movq %rcx, %xmm0
+; SSE2-NEXT: movq %rsi, %xmm4
+; SSE2-NEXT: movq %rdi, %xmm1
+; SSE2-NEXT: movq %r8, %xmm6
+; SSE2-NEXT: movq %r9, %xmm2
+; SSE2-NEXT: punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm4[0]
+; SSE2-NEXT: punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm6[0]
+; SSE2-NEXT: movq %r10, %xmm4
+; SSE2-NEXT: punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm4[0]
+; SSE2-NEXT: movq %rax, %xmm4
+; SSE2-NEXT: pshufd {{.*#+}} xmm5 = xmm5[2,3,2,3]
+; SSE2-NEXT: movq %xmm5, %rcx
+; SSE2-NEXT: pshufd {{.*#+}} xmm3 = xmm3[2,3,2,3]
+; SSE2-NEXT: movq %xmm3, %rax
; SSE2-NEXT: xorl %edx, %edx
; SSE2-NEXT: divq %rcx
-; SSE2-NEXT: movq %rax, %xmm1
-; SSE2-NEXT: punpcklqdq {{.*#+}} xmm6 = xmm6[0],xmm1[0]
-; SSE2-NEXT: movdqa %xmm4, %xmm1
-; SSE2-NEXT: movdqa %xmm5, %xmm2
-; SSE2-NEXT: movdqa %xmm6, %xmm3
+; SSE2-NEXT: movq %rax, %xmm3
+; SSE2-NEXT: punpcklqdq {{.*#+}} xmm4 = xmm4[0],xmm3[0]
+; SSE2-NEXT: movdqa %xmm4, %xmm3
; SSE2-NEXT: retq
;
; SSE42-LABEL: udiv_v8i64:
; SSE42: # %bb.0:
-; SSE42-NEXT: movdqa %xmm0, %xmm11
-; SSE42-NEXT: movdqa {{[0-9]+}}(%rsp), %xmm0
-; SSE42-NEXT: pshufd {{.*#+}} xmm8 = xmm0[3,3,3,3]
-; SSE42-NEXT: pmovzxwq {{.*#+}} xmm8 = xmm8[0],zero,zero,zero,xmm8[1],zero,zero,zero
-; SSE42-NEXT: psllq $63, %xmm8
-; SSE42-NEXT: pshufd {{.*#+}} xmm9 = xmm0[2,3,2,3]
-; SSE42-NEXT: pmovzxwq {{.*#+}} xmm9 = xmm9[0],zero,zero,zero,xmm9[1],zero,zero,zero
-; SSE42-NEXT: psllq $63, %xmm9
-; SSE42-NEXT: pshufd {{.*#+}} xmm10 = xmm0[1,1,1,1]
-; SSE42-NEXT: pmovzxwq {{.*#+}} xmm10 = xmm10[0],zero,zero,zero,xmm10[1],zero,zero,zero
-; SSE42-NEXT: pmovzxwq {{.*#+}} xmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero
+; SSE42-NEXT: pushq %rbx
+; SSE42-NEXT: .cfi_def_cfa_offset 16
+; SSE42-NEXT: .cfi_offset %rbx, -16
+; SSE42-NEXT: movdqa %xmm0, %xmm8
+; SSE42-NEXT: movdqa {{[0-9]+}}(%rsp), %xmm10
+; SSE42-NEXT: pmovzxwq {{.*#+}} xmm0 = xmm10[0],zero,zero,zero,xmm10[1],zero,zero,zero
; SSE42-NEXT: psllq $63, %xmm0
-; SSE42-NEXT: movapd {{.*#+}} xmm12 = [1,1]
-; SSE42-NEXT: movapd %xmm12, %xmm13
-; SSE42-NEXT: blendvpd %xmm0, %xmm4, %xmm13
-; SSE42-NEXT: pextrq $1, %xmm13, %rcx
-; SSE42-NEXT: pextrq $1, %xmm11, %rax
-; SSE42-NEXT: psllq $63, %xmm10
+; SSE42-NEXT: movapd {{.*#+}} xmm9 = [1,1]
+; SSE42-NEXT: movapd %xmm9, %xmm11
+; SSE42-NEXT: blendvpd %xmm0, %xmm4, %xmm11
+; SSE42-NEXT: pextrq $1, %xmm11, %rcx
+; SSE42-NEXT: pextrq $1, %xmm8, %rax
; SSE42-NEXT: xorl %edx, %edx
; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm0
-; SSE42-NEXT: movq %xmm13, %rcx
-; SSE42-NEXT: movq %xmm11, %rax
+; SSE42-NEXT: movq %rax, %rcx
+; SSE42-NEXT: movq %xmm11, %rsi
+; SSE42-NEXT: movq %xmm8, %rax
; SSE42-NEXT: xorl %edx, %edx
-; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm4
-; SSE42-NEXT: punpcklqdq {{.*#+}} xmm4 = xmm4[0],xmm0[0]
-; SSE42-NEXT: movapd %xmm12, %xmm11
-; SSE42-NEXT: movdqa %xmm10, %xmm0
-; SSE42-NEXT: blendvpd %xmm0, %xmm5, %xmm11
-; SSE42-NEXT: pextrq $1, %xmm11, %rcx
+; SSE42-NEXT: divq %rsi
+; SSE42-NEXT: movq %rax, %rsi
+; SSE42-NEXT: pshufd {{.*#+}} xmm0 = xmm10[1,1,1,1]
+; SSE42-NEXT: pmovzxwq {{.*#+}} xmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero
+; SSE42-NEXT: psllq $63, %xmm0
+; SSE42-NEXT: movapd %xmm9, %xmm4
+; SSE42-NEXT: blendvpd %xmm0, %xmm5, %xmm4
+; SSE42-NEXT: pextrq $1, %xmm4, %rdi
; SSE42-NEXT: pextrq $1, %xmm1, %rax
; SSE42-NEXT: xorl %edx, %edx
-; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm5
-; SSE42-NEXT: movq %xmm11, %rcx
+; SSE42-NEXT: divq %rdi
+; SSE42-NEXT: movq %rax, %rdi
+; SSE42-NEXT: pshufd {{.*#+}} xmm0 = xmm10[2,3,2,3]
+; SSE42-NEXT: pmovzxwq {{.*#+}} xmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero
+; SSE42-NEXT: psllq $63, %xmm0
+; SSE42-NEXT: movq %xmm4, %r8
+; SSE42-NEXT: movapd %xmm9, %xmm4
+; SSE42-NEXT: blendvpd %xmm0, %xmm6, %xmm4
+; SSE42-NEXT: pextrq $1, %xmm4, %r10
+; SSE42-NEXT: pextrq $1, %xmm2, %r9
; SSE42-NEXT: movq %xmm1, %rax
; SSE42-NEXT: xorl %edx, %edx
-; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm1
-; SSE42-NEXT: movapd %xmm12, %xmm10
-; SSE42-NEXT: movdqa %xmm9, %xmm0
-; SSE42-NEXT: blendvpd %xmm0, %xmm6, %xmm10
-; SSE42-NEXT: pextrq $1, %xmm10, %rcx
-; SSE42-NEXT: punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm5[0]
-; SSE42-NEXT: pextrq $1, %xmm2, %rax
+; SSE42-NEXT: divq %r8
+; SSE42-NEXT: movq %rax, %r8
+; SSE42-NEXT: movq %r9, %rax
; SSE42-NEXT: xorl %edx, %edx
-; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm5
-; SSE42-NEXT: movq %xmm10, %rcx
+; SSE42-NEXT: divq %r10
+; SSE42-NEXT: movq %rax, %r9
+; SSE42-NEXT: pshufd {{.*#+}} xmm0 = xmm10[3,3,3,3]
+; SSE42-NEXT: pmovzxwq {{.*#+}} xmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero
+; SSE42-NEXT: psllq $63, %xmm0
+; SSE42-NEXT: movq %xmm4, %r11
+; SSE42-NEXT: blendvpd %xmm0, %xmm7, %xmm9
+; SSE42-NEXT: pextrq $1, %xmm9, %rbx
+; SSE42-NEXT: pextrq $1, %xmm3, %r10
; SSE42-NEXT: movq %xmm2, %rax
; SSE42-NEXT: xorl %edx, %edx
-; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm2
-; SSE42-NEXT: movdqa %xmm8, %xmm0
-; SSE42-NEXT: blendvpd %xmm0, %xmm7, %xmm12
-; SSE42-NEXT: pextrq $1, %xmm12, %rcx
-; SSE42-NEXT: punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm5[0]
-; SSE42-NEXT: pextrq $1, %xmm3, %rax
+; SSE42-NEXT: divq %r11
+; SSE42-NEXT: movq %rax, %r11
+; SSE42-NEXT: movq %r10, %rax
; SSE42-NEXT: xorl %edx, %edx
-; SSE42-NEXT: divq %rcx
-; SSE42-NEXT: movq %rax, %xmm0
-; SSE42-NEXT: movq %xmm12, %rcx
+; SSE42-NEXT: divq %rbx
+; SSE42-NEXT: movq %rcx, %xmm2
+; SSE42-NEXT: movq %rsi, %xmm0
+; SSE42-NEXT: movq %rdi, %xmm4
+; SSE42-NEXT: movq %r8, %xmm1
+; SSE42-NEXT: movq %r9, %xmm5
+; SSE42-NEXT: punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm2[0]
+; SSE42-NEXT: punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm4[0]
+; SSE42-NEXT: movq %r11, %xmm2
+; SSE42-NEXT: punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm5[0]
+; SSE42-NEXT: movq %rax, %xmm4
+; SSE42-NEXT: movq %xmm9, %rcx
; SSE42-NEXT: movq %xmm3, %rax
; SSE42-NEXT: xorl %edx, %edx
; SSE42-NEXT: divq %rcx
; SSE42-NEXT: movq %rax, %xmm3
-; SSE42-NEXT: punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm0[0]
-; SSE42-NEXT: movdqa %xmm4, %xmm0
+; SSE42-NEXT: punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm4[0]
+; SSE42-NEXT: popq %rbx
+; SSE42-NEXT: .cfi_def_cfa_offset 8
; SSE42-NEXT: retq
;
; AVX2-LABEL: udiv_v8i64:
@@ -995,55 +1010,57 @@ define <8 x i64> @udiv_v8i64(<8 x i64> %x, <8 x i64> %y, <8 x i1> %m) {
; AVX2-NEXT: xorl %edx, %edx
; AVX2-NEXT: divq %rcx
; AVX2-NEXT: movq %rax, %rcx
-; AVX2-NEXT: vpunpckhwd {{.*#+}} xmm4 = xmm4[4,4,5,5,6,6,7,7]
; AVX2-NEXT: vmovq %xmm6, %rsi
; AVX2-NEXT: vmovq %xmm7, %rax
; AVX2-NEXT: xorl %edx, %edx
; AVX2-NEXT: divq %rsi
; AVX2-NEXT: movq %rax, %rsi
-; AVX2-NEXT: vpslld $31, %xmm4, %xmm4
-; AVX2-NEXT: vpmovsxdq %xmm4, %ymm4
; AVX2-NEXT: vpextrq $1, %xmm2, %rdi
-; AVX2-NEXT: vmovq %rcx, %xmm6
; AVX2-NEXT: vpextrq $1, %xmm0, %rax
; AVX2-NEXT: xorl %edx, %edx
; AVX2-NEXT: divq %rdi
-; AVX2-NEXT: movq %rax, %rcx
-; AVX2-NEXT: vmovq %rsi, %xmm7
-; AVX2-NEXT: vmovq %xmm2, %rsi
+; AVX2-NEXT: movq %rax, %rdi
+; AVX2-NEXT: vmovq %xmm2, %r8
; AVX2-NEXT: vmovq %xmm0, %rax
; AVX2-NEXT: xorl %edx, %edx
-; AVX2-NEXT: divq %rsi
-; AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm0 = xmm7[0],xmm6[0]
-; AVX2-NEXT: vmovq %rcx, %xmm6
-; AVX2-NEXT: vblendvpd %ymm4, %ymm3, %ymm5, %ymm2
-; AVX2-NEXT: vmovq %rax, %xmm3
-; AVX2-NEXT: vextractf128 $1, %ymm2, %xmm4
-; AVX2-NEXT: vpextrq $1, %xmm4, %rcx
-; AVX2-NEXT: vextracti128 $1, %ymm1, %xmm5
-; AVX2-NEXT: vpextrq $1, %xmm5, %rax
+; AVX2-NEXT: divq %r8
+; AVX2-NEXT: movq %rax, %r8
+; AVX2-NEXT: vpunpckhwd {{.*#+}} xmm0 = xmm4[4,4,5,5,6,6,7,7]
+; AVX2-NEXT: vpslld $31, %xmm0, %xmm0
+; AVX2-NEXT: vpmovsxdq %xmm0, %ymm0
+; AVX2-NEXT: vblendvpd %ymm0, %ymm3, %ymm5, %ymm2
+; AVX2-NEXT: vextractf128 $1, %ymm2, %xmm0
+; AVX2-NEXT: vpextrq $1, %xmm0, %r9
+; AVX2-NEXT: vextracti128 $1, %ymm1, %xmm3
+; AVX2-NEXT: vpextrq $1, %xmm3, %rax
; AVX2-NEXT: xorl %edx, %edx
-; AVX2-NEXT: divq %rcx
-; AVX2-NEXT: movq %rax, %rcx
-; AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm6[0]
-; AVX2-NEXT: vmovq %xmm4, %rsi
-; AVX2-NEXT: vmovq %xmm5, %rax
+; AVX2-NEXT: divq %r9
+; AVX2-NEXT: movq %rax, %r9
+; AVX2-NEXT: vmovq %xmm0, %r10
+; AVX2-NEXT: vmovq %xmm3, %rax
; AVX2-NEXT: xorl %edx, %edx
-; AVX2-NEXT: divq %rsi
-; AVX2-NEXT: vinserti128 $1, %xmm0, %ymm3, %ymm0
-; AVX2-NEXT: vmovq %rcx, %xmm3
-; AVX2-NEXT: vpextrq $1, %xmm2, %rcx
-; AVX2-NEXT: vmovq %rax, %xmm4
+; AVX2-NEXT: divq %r10
+; AVX2-NEXT: movq %rax, %r10
+; AVX2-NEXT: vpextrq $1, %xmm2, %r11
; AVX2-NEXT: vpextrq $1, %xmm1, %rax
; AVX2-NEXT: xorl %edx, %edx
-; AVX2-NEXT: divq %rcx
-; AVX2-NEXT: movq %rax, %rcx
+; AVX2-NEXT: divq %r11
+; AVX2-NEXT: movq %rax, %r11
+; AVX2-NEXT: vmovq %rcx, %xmm0
+; AVX2-NEXT: vmovq %rsi, %xmm3
+; AVX2-NEXT: vmovq %rdi, %xmm4
+; AVX2-NEXT: vmovq %r8, %xmm5
+; AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm0 = xmm3[0],xmm0[0]
+; AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm5[0],xmm4[0]
+; AVX2-NEXT: vinserti128 $1, %xmm0, %ymm3, %ymm0
+; AVX2-NEXT: vmovq %r9, %xmm3
+; AVX2-NEXT: vmovq %r10, %xmm4
; AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm4[0],xmm3[0]
-; AVX2-NEXT: vmovq %xmm2, %rsi
+; AVX2-NEXT: vmovq %xmm2, %rcx
; AVX2-NEXT: vmovq %xmm1, %rax
; AVX2-NEXT: xorl %edx, %edx
-; AVX2-NEXT: divq %rsi
-; AVX2-NEXT: vmovq %rcx, %xmm1
+; AVX2-NEXT: divq %rcx
+; AVX2-NEXT: vmovq %r11, %xmm1
; AVX2-NEXT: vmovq %rax, %xmm2
; AVX2-NEXT: vpunpcklqdq {{.*#+}} xmm1 = xmm2[0],xmm1[0]
; AVX2-NEXT: vinserti128 $1, %xmm3, %ymm1, %ymm1
diff --git a/llvm/test/CodeGen/X86/min-legal-vector-width.ll b/llvm/test/CodeGen/X86/min-legal-vector-width.ll
index 5d80435618dd5..b4f5a50ba1b46 100644
--- a/llvm/test/CodeGen/X86/min-legal-vector-width.ll
+++ b/llvm/test/CodeGen/X86/min-legal-vector-width.ll
@@ -2072,54 +2072,54 @@ define <8 x i64> @udiv_v8i64_prefer256(<8 x i64> %x, <8 x i64> %y) nounwind "min
; CHECK-NEXT: vpextrq $1, %xmm5, %rcx
; CHECK-NEXT: xorl %edx, %edx
; CHECK-NEXT: divq %rcx
-; CHECK-NEXT: movq %rax, %rdi
-; CHECK-NEXT: vmovq %xmm4, %rax
-; CHECK-NEXT: vmovq %xmm5, %rcx
-; CHECK-NEXT: xorl %edx, %edx
-; CHECK-NEXT: divq %rcx
; CHECK-NEXT: movq %rax, %rcx
-; CHECK-NEXT: vpextrq $1, %xmm0, %rax
-; CHECK-NEXT: vpextrq $1, %xmm2, %rsi
+; CHECK-NEXT: vmovq %xmm4, %rax
+; CHECK-NEXT: vmovq %xmm5, %rsi
; CHECK-NEXT: xorl %edx, %edx
; CHECK-NEXT: divq %rsi
; CHECK-NEXT: movq %rax, %rsi
+; CHECK-NEXT: vpextrq $1, %xmm0, %rax
+; CHECK-NEXT: vpextrq $1, %xmm2, %rdi
+; CHECK-NEXT: xorl %edx, %edx
+; CHECK-NEXT: divq %rdi
+; CHECK-NEXT: movq %rax, %rdi
; CHECK-NEXT: vmovq %xmm0, %rax
; CHECK-NEXT: vmovq %xmm2, %r8
; CHECK-NEXT: xorl %edx, %edx
; CHECK-NEXT: divq %r8
; CHECK-NEXT: movq %rax, %r8
-; CHECK-NEXT: vmovq %rdi, %xmm0
-; CHECK-NEXT: vextracti128 $1, %ymm1, %xmm2
-; CHECK-NEXT: vpextrq $1, %xmm2, %rax
-; CHECK-NEXT: vextracti128 $1, %ymm3, %xmm4
-; CHECK-NEXT: vpextrq $1, %xmm4, %rdi
+; CHECK-NEXT: vextracti128 $1, %ymm1, %xmm0
+; CHECK-NEXT: vpextrq $1, %xmm0, %rax
+; CHECK-NEXT: vextracti128 $1, %ymm3, %xmm2
+; CHECK-NEXT: vpextrq $1, %xmm2, %r9
; CHECK-NEXT: xorl %edx, %edx
-; CHECK-NEXT: divq %rdi
-; CHECK-NEXT: movq %rax, %rdi
-; CHECK-NEXT: vmovq %rcx, %xmm5
-; CHECK-NEXT: vmovq %xmm2, %rax
-; CHECK-NEXT: vmovq %xmm4, %rcx
+; CHECK-NEXT: divq %r9
+; CHECK-NEXT: movq %rax, %r9
+; CHECK-NEXT: vmovq %xmm0, %rax
+; CHECK-NEXT: vmovq %xmm2, %r10
; CHECK-NEXT: xorl %edx, %edx
-; CHECK-NEXT: divq %rcx
-; CHECK-NEXT: movq %rax, %rcx
-; CHECK-NEXT: vpunpcklqdq {{.*#+}} xmm0 = xmm5[0],xmm0[0]
-; CHECK-NEXT: vmovq %rsi, %xmm2
-; CHECK-NEXT: vmovq %r8, %xmm4
-; CHECK-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm4[0],xmm2[0]
+; CHECK-NEXT: divq %r10
+; CHECK-NEXT: movq %rax, %r10
; CHECK-NEXT: vpextrq $1, %xmm1, %rax
-; CHECK-NEXT: vinserti128 $1, %xmm0, %ymm2, %ymm0
-; CHECK-NEXT: vpextrq $1, %xmm3, %rsi
+; CHECK-NEXT: vpextrq $1, %xmm3, %r11
; CHECK-NEXT: xorl %edx, %edx
-; CHECK-NEXT: divq %rsi
-; CHECK-NEXT: movq %rax, %rsi
-; CHECK-NEXT: vmovq %rdi, %xmm2
+; CHECK-NEXT: divq %r11
+; CHECK-NEXT: movq %rax, %r11
+; CHECK-NEXT: vmovq %rcx, %xmm0
+; CHECK-NEXT: vmovq %rsi, %xmm2
+; CHECK-NEXT: vmovq %rdi, %xmm4
+; CHECK-NEXT: vpunpcklqdq {{.*#+}} xmm0 = xmm2[0],xmm0[0]
+; CHECK-NEXT: vmovq %r8, %xmm2
+; CHECK-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm4[0]
+; CHECK-NEXT: vinserti128 $1, %xmm0, %ymm2, %ymm0
+; CHECK-NEXT: vmovq %r9, %xmm2
; CHECK-NEXT: vmovq %xmm1, %rax
-; CHECK-NEXT: vmovq %xmm3, %rdi
+; CHECK-NEXT: vmovq %xmm3, %rcx
; CHECK-NEXT: xorl %edx, %edx
-; CHECK-NEXT: divq %rdi
-; CHECK-NEXT: vmovq %rcx, %xmm1
+; CHECK-NEXT: divq %rcx
+; CHECK-NEXT: vmovq %r10, %xmm1
; CHECK-NEXT: vpunpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm2[0]
-; CHECK-NEXT: vmovq %rsi, %xmm2
+; CHECK-NEXT: vmovq %r11, %xmm2
; CHECK-NEXT: vmovq %rax, %xmm3
; CHECK-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm3[0],xmm2[0]
; CHECK-NEXT: vinserti128 $1, %xmm1, %ymm2, %ymm1
diff --git a/llvm/test/CodeGen/X86/pr205272.ll b/llvm/test/CodeGen/X86/pr205272.ll
new file mode 100644
index 0000000000000..1cfdae2dcc110
--- /dev/null
+++ b/llvm/test/CodeGen/X86/pr205272.ll
@@ -0,0 +1,23 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s
+
+declare i64 @llvm.x86.rdpmc(i32) #0
+declare i64 @llvm.x86.rdpru(i32) #0
+
+define i64 @f(i32 %0) #1 {
+; CHECK-LABEL: f:
+; CHECK: # %bb.0:
+; CHECK-NEXT: xorl %ecx, %ecx
+; CHECK-NEXT: rdpmc
+; CHECK-NEXT: movl %edi, %ecx
+; CHECK-NEXT: rdpru
+; CHECK-NEXT: shlq $32, %rdx
+; CHECK-NEXT: orq %rdx, %rax
+; CHECK-NEXT: retq
+ %2 = tail call i64 @llvm.x86.rdpmc(i32 0)
+ %3 = tail call i64 @llvm.x86.rdpru(i32 %0)
+ ret i64 %3
+}
+
+attributes #0 = { nounwind }
+attributes #1 = { "target-features"="+avx512f" }
diff --git a/llvm/test/CodeGen/X86/ssub_sat_plus.ll b/llvm/test/CodeGen/X86/ssub_sat_plus.ll
index 5baf7a1dac74c..8b96a8050e65e 100644
--- a/llvm/test/CodeGen/X86/ssub_sat_plus.ll
+++ b/llvm/test/CodeGen/X86/ssub_sat_plus.ll
@@ -105,9 +105,9 @@ define signext i16 @func16(i16 signext %x, i16 signext %y, i16 signext %z) nounw
define signext i8 @func8(i8 signext %x, i8 signext %y, i8 signext %z) nounwind {
; X86-LABEL: func8:
; X86: # %bb.0:
-; X86-NEXT: movzbl {{[0-9]+}}(%esp), %edx
; X86-NEXT: movzbl {{[0-9]+}}(%esp), %eax
; X86-NEXT: mulb {{[0-9]+}}(%esp)
+; X86-NEXT: movzbl {{[0-9]+}}(%esp), %edx
; X86-NEXT: xorl %ecx, %ecx
; X86-NEXT: cmpb %al, %dl
; X86-NEXT: setns %cl
@@ -140,9 +140,9 @@ define signext i8 @func8(i8 signext %x, i8 signext %y, i8 signext %z) nounwind {
define signext i4 @func4(i4 signext %x, i4 signext %y, i4 signext %z) nounwind {
; X86-LABEL: func4:
; X86: # %bb.0:
-; X86-NEXT: movzbl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movzbl {{[0-9]+}}(%esp), %eax
; X86-NEXT: mulb {{[0-9]+}}(%esp)
+; X86-NEXT: movzbl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: shlb $4, %al
; X86-NEXT: sarb $4, %al
; X86-NEXT: subb %al, %cl
diff --git a/llvm/test/CodeGen/X86/udiv_fix.ll b/llvm/test/CodeGen/X86/udiv_fix.ll
index b2ad846b1be7e..8c3698c535b00 100644
--- a/llvm/test/CodeGen/X86/udiv_fix.ll
+++ b/llvm/test/CodeGen/X86/udiv_fix.ll
@@ -230,31 +230,33 @@ define <4 x i32> @vec(<4 x i32> %x, <4 x i32> %y) nounwind {
; X64-LABEL: vec:
; X64: # %bb.0:
; X64-NEXT: pxor %xmm2, %xmm2
-; X64-NEXT: movdqa %xmm0, %xmm4
-; X64-NEXT: punpckhdq {{.*#+}} xmm4 = xmm4[2],xmm2[2],xmm4[3],xmm2[3]
-; X64-NEXT: psllq $31, %xmm4
-; X64-NEXT: movq %xmm4, %rax
-; X64-NEXT: movdqa %xmm1, %xmm3
+; X64-NEXT: movdqa %xmm0, %xmm3
; X64-NEXT: punpckhdq {{.*#+}} xmm3 = xmm3[2],xmm2[2],xmm3[3],xmm2[3]
-; X64-NEXT: movd %xmm3, %ecx
-; X64-NEXT: xorl %edx, %edx
-; X64-NEXT: divq %rcx
-; X64-NEXT: movq %rax, %xmm3
-; X64-NEXT: pshufd {{.*#+}} xmm4 = xmm4[2,3,2,3]
-; X64-NEXT: movq %xmm4, %rax
+; X64-NEXT: psllq $31, %xmm3
+; X64-NEXT: movq %xmm3, %rax
; X64-NEXT: movdqa %xmm1, %xmm4
-; X64-NEXT: psrldq {{.*#+}} xmm4 = xmm4[12,13,14,15],zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero
+; X64-NEXT: punpckhdq {{.*#+}} xmm4 = xmm4[2],xmm2[2],xmm4[3],xmm2[3]
; X64-NEXT: movd %xmm4, %ecx
; X64-NEXT: xorl %edx, %edx
; X64-NEXT: divq %rcx
-; X64-NEXT: movq %rax, %xmm4
-; X64-NEXT: punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm4[0]
+; X64-NEXT: movq %rax, %rcx
+; X64-NEXT: pshufd {{.*#+}} xmm3 = xmm3[2,3,2,3]
+; X64-NEXT: movq %xmm3, %rax
+; X64-NEXT: movdqa %xmm1, %xmm3
+; X64-NEXT: psrldq {{.*#+}} xmm3 = xmm3[12,13,14,15],zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero
+; X64-NEXT: movd %xmm3, %esi
+; X64-NEXT: xorl %edx, %edx
+; X64-NEXT: divq %rsi
+; X64-NEXT: movq %rax, %rsi
; X64-NEXT: punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
; X64-NEXT: psllq $31, %xmm0
; X64-NEXT: movq %xmm0, %rax
-; X64-NEXT: movd %xmm1, %ecx
+; X64-NEXT: movd %xmm1, %edi
; X64-NEXT: xorl %edx, %edx
-; X64-NEXT: divq %rcx
+; X64-NEXT: divq %rdi
+; X64-NEXT: movq %rcx, %xmm3
+; X64-NEXT: movq %rsi, %xmm2
+; X64-NEXT: punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm2[0]
; X64-NEXT: movq %rax, %xmm2
; X64-NEXT: pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
; X64-NEXT: movq %xmm0, %rax
diff --git a/llvm/test/CodeGen/X86/udiv_fix_sat.ll b/llvm/test/CodeGen/X86/udiv_fix_sat.ll
index 565918e5b159b..659ab7e69e9ae 100644
--- a/llvm/test/CodeGen/X86/udiv_fix_sat.ll
+++ b/llvm/test/CodeGen/X86/udiv_fix_sat.ll
@@ -302,59 +302,61 @@ define <4 x i32> @vec(<4 x i32> %x, <4 x i32> %y) nounwind {
; X64-NEXT: movd %xmm4, %ecx
; X64-NEXT: xorl %edx, %edx
; X64-NEXT: divq %rcx
-; X64-NEXT: movq %rax, %xmm8
+; X64-NEXT: movq %rax, %rcx
; X64-NEXT: pshufd {{.*#+}} xmm3 = xmm3[2,3,2,3]
; X64-NEXT: movq %xmm3, %rax
; X64-NEXT: movdqa %xmm1, %xmm3
; X64-NEXT: psrldq {{.*#+}} xmm3 = xmm3[12,13,14,15],zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero
-; X64-NEXT: movd %xmm3, %ecx
+; X64-NEXT: movd %xmm3, %esi
; X64-NEXT: xorl %edx, %edx
-; X64-NEXT: divq %rcx
-; X64-NEXT: movq %rax, %xmm3
-; X64-NEXT: punpcklqdq {{.*#+}} xmm8 = xmm8[0],xmm3[0]
-; X64-NEXT: movdqa {{.*#+}} xmm4 = [9223372039002259456,9223372039002259456]
-; X64-NEXT: movdqa %xmm8, %xmm3
-; X64-NEXT: pxor %xmm4, %xmm3
-; X64-NEXT: pshufd {{.*#+}} xmm6 = xmm3[1,1,3,3]
-; X64-NEXT: movdqa {{.*#+}} xmm7 = [2147483649,2147483649,2147483649,2147483649]
-; X64-NEXT: pcmpeqd %xmm7, %xmm6
-; X64-NEXT: movdqa {{.*#+}} xmm5 = [9223372043297226751,9223372043297226751]
-; X64-NEXT: movdqa %xmm5, %xmm9
-; X64-NEXT: pcmpgtd %xmm3, %xmm9
-; X64-NEXT: pshufd {{.*#+}} xmm10 = xmm9[0,0,2,2]
-; X64-NEXT: pand %xmm6, %xmm10
-; X64-NEXT: pshufd {{.*#+}} xmm3 = xmm9[1,1,3,3]
-; X64-NEXT: por %xmm10, %xmm3
-; X64-NEXT: pcmpeqd %xmm6, %xmm6
-; X64-NEXT: pand %xmm3, %xmm8
-; X64-NEXT: pxor %xmm6, %xmm3
-; X64-NEXT: por %xmm8, %xmm3
-; X64-NEXT: psrlq $1, %xmm3
+; X64-NEXT: divq %rsi
+; X64-NEXT: movq %rax, %rsi
; X64-NEXT: punpckldq {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1]
; X64-NEXT: movq %xmm2, %rax
-; X64-NEXT: movd %xmm1, %ecx
+; X64-NEXT: movd %xmm1, %edi
; X64-NEXT: xorl %edx, %edx
-; X64-NEXT: divq %rcx
-; X64-NEXT: movq %rax, %xmm8
-; X64-NEXT: pshufd {{.*#+}} xmm0 = xmm2[2,3,2,3]
-; X64-NEXT: movq %xmm0, %rax
+; X64-NEXT: divq %rdi
+; X64-NEXT: movq %rcx, %xmm7
+; X64-NEXT: movq %rsi, %xmm0
+; X64-NEXT: punpcklqdq {{.*#+}} xmm7 = xmm7[0],xmm0[0]
+; X64-NEXT: movdqa {{.*#+}} xmm0 = [9223372039002259456,9223372039002259456]
+; X64-NEXT: movdqa %xmm7, %xmm3
+; X64-NEXT: pxor %xmm0, %xmm3
+; X64-NEXT: pshufd {{.*#+}} xmm4 = xmm3[1,1,3,3]
+; X64-NEXT: movdqa {{.*#+}} xmm6 = [2147483649,2147483649,2147483649,2147483649]
+; X64-NEXT: pcmpeqd %xmm6, %xmm4
+; X64-NEXT: movdqa {{.*#+}} xmm5 = [9223372043297226751,9223372043297226751]
+; X64-NEXT: movdqa %xmm5, %xmm8
+; X64-NEXT: pcmpgtd %xmm3, %xmm8
+; X64-NEXT: pshufd {{.*#+}} xmm9 = xmm8[0,0,2,2]
+; X64-NEXT: pand %xmm4, %xmm9
+; X64-NEXT: pshufd {{.*#+}} xmm3 = xmm8[1,1,3,3]
+; X64-NEXT: por %xmm9, %xmm3
+; X64-NEXT: pcmpeqd %xmm4, %xmm4
+; X64-NEXT: pand %xmm3, %xmm7
+; X64-NEXT: pxor %xmm4, %xmm3
+; X64-NEXT: por %xmm7, %xmm3
+; X64-NEXT: psrlq $1, %xmm3
+; X64-NEXT: movq %rax, %xmm7
+; X64-NEXT: pshufd {{.*#+}} xmm2 = xmm2[2,3,2,3]
+; X64-NEXT: movq %xmm2, %rax
; X64-NEXT: psrlq $32, %xmm1
; X64-NEXT: movd %xmm1, %ecx
; X64-NEXT: xorl %edx, %edx
; X64-NEXT: divq %rcx
-; X64-NEXT: movq %rax, %xmm0
-; X64-NEXT: punpcklqdq {{.*#+}} xmm8 = xmm8[0],xmm0[0]
-; X64-NEXT: pxor %xmm8, %xmm4
-; X64-NEXT: pshufd {{.*#+}} xmm0 = xmm4[1,1,3,3]
-; X64-NEXT: pcmpeqd %xmm7, %xmm0
-; X64-NEXT: pcmpgtd %xmm4, %xmm5
-; X64-NEXT: pshufd {{.*#+}} xmm1 = xmm5[0,0,2,2]
-; X64-NEXT: pand %xmm0, %xmm1
+; X64-NEXT: movq %rax, %xmm1
+; X64-NEXT: punpcklqdq {{.*#+}} xmm7 = xmm7[0],xmm1[0]
+; X64-NEXT: pxor %xmm7, %xmm0
+; X64-NEXT: pshufd {{.*#+}} xmm1 = xmm0[1,1,3,3]
+; X64-NEXT: pcmpeqd %xmm6, %xmm1
+; X64-NEXT: pcmpgtd %xmm0, %xmm5
+; X64-NEXT: pshufd {{.*#+}} xmm2 = xmm5[0,0,2,2]
+; X64-NEXT: pand %xmm1, %xmm2
; X64-NEXT: pshufd {{.*#+}} xmm0 = xmm5[1,1,3,3]
-; X64-NEXT: por %xmm1, %xmm0
-; X64-NEXT: pxor %xmm0, %xmm6
-; X64-NEXT: pand %xmm8, %xmm0
-; X64-NEXT: por %xmm6, %xmm0
+; X64-NEXT: por %xmm2, %xmm0
+; X64-NEXT: pxor %xmm0, %xmm4
+; X64-NEXT: pand %xmm7, %xmm0
+; X64-NEXT: por %xmm4, %xmm0
; X64-NEXT: psrlq $1, %xmm0
; X64-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,2],xmm3[0,2]
; X64-NEXT: retq
diff --git a/llvm/test/CodeGen/X86/usub_sat_plus.ll b/llvm/test/CodeGen/X86/usub_sat_plus.ll
index 0fb14ad5cf7b0..725de4401cb87 100644
--- a/llvm/test/CodeGen/X86/usub_sat_plus.ll
+++ b/llvm/test/CodeGen/X86/usub_sat_plus.ll
@@ -82,9 +82,9 @@ define zeroext i16 @func16(i16 zeroext %x, i16 zeroext %y, i16 zeroext %z) nounw
define zeroext i8 @func8(i8 zeroext %x, i8 zeroext %y, i8 zeroext %z) nounwind {
; X86-LABEL: func8:
; X86: # %bb.0:
-; X86-NEXT: movzbl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movzbl {{[0-9]+}}(%esp), %eax
; X86-NEXT: mulb {{[0-9]+}}(%esp)
+; X86-NEXT: movzbl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: xorl %edx, %edx
; X86-NEXT: subb %al, %cl
; X86-NEXT: movzbl %cl, %eax
@@ -111,9 +111,9 @@ define zeroext i8 @func8(i8 zeroext %x, i8 zeroext %y, i8 zeroext %z) nounwind {
define zeroext i4 @func4(i4 zeroext %x, i4 zeroext %y, i4 zeroext %z) nounwind {
; X86-LABEL: func4:
; X86: # %bb.0:
-; X86-NEXT: movzbl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movzbl {{[0-9]+}}(%esp), %eax
; X86-NEXT: mulb {{[0-9]+}}(%esp)
+; X86-NEXT: movzbl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: andb $15, %al
; X86-NEXT: xorl %edx, %edx
; X86-NEXT: subb %al, %cl
diff --git a/llvm/test/CodeGen/X86/vector-idiv-strictfp.ll b/llvm/test/CodeGen/X86/vector-idiv-strictfp.ll
index 972e759d1763c..7e4fbaec1019e 100644
--- a/llvm/test/CodeGen/X86/vector-idiv-strictfp.ll
+++ b/llvm/test/CodeGen/X86/vector-idiv-strictfp.ll
@@ -293,44 +293,50 @@ define <8 x i64> @test_divv_8i64_strictfp(<8 x i64> %a, <8 x i64> %b) nounwind s
; NODQ-NEXT: vpextrq $1, %xmm3, %rcx
; NODQ-NEXT: xorl %edx, %edx
; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm4
+; NODQ-NEXT: movq %rax, %rcx
; NODQ-NEXT: vmovq %xmm2, %rax
-; NODQ-NEXT: vmovq %xmm3, %rcx
+; NODQ-NEXT: vmovq %xmm3, %rsi
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm2
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm4[0]
-; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm3
-; NODQ-NEXT: vpextrq $1, %xmm3, %rax
-; NODQ-NEXT: vextracti32x4 $2, %zmm1, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rcx
+; NODQ-NEXT: divq %rsi
+; NODQ-NEXT: movq %rax, %rsi
+; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm2
+; NODQ-NEXT: vpextrq $1, %xmm2, %rax
+; NODQ-NEXT: vextracti32x4 $2, %zmm1, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %rdi
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm5
-; NODQ-NEXT: vmovq %xmm3, %rax
-; NODQ-NEXT: vmovq %xmm4, %rcx
+; NODQ-NEXT: divq %rdi
+; NODQ-NEXT: movq %rax, %rdi
+; NODQ-NEXT: vmovq %xmm2, %rax
+; NODQ-NEXT: vmovq %xmm3, %r8
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
-; NODQ-NEXT: vinserti128 $1, %xmm2, %ymm3, %ymm2
-; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm3
-; NODQ-NEXT: vpextrq $1, %xmm3, %rax
-; NODQ-NEXT: vextracti128 $1, %ymm1, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rcx
+; NODQ-NEXT: divq %r8
+; NODQ-NEXT: movq %rax, %r8
+; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm2
+; NODQ-NEXT: vpextrq $1, %xmm2, %rax
+; NODQ-NEXT: vextracti128 $1, %ymm1, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %r9
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm5
-; NODQ-NEXT: vmovq %xmm3, %rax
-; NODQ-NEXT: vmovq %xmm4, %rcx
+; NODQ-NEXT: divq %r9
+; NODQ-NEXT: movq %rax, %r9
+; NODQ-NEXT: vmovq %xmm2, %rax
+; NODQ-NEXT: vmovq %xmm3, %r10
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
+; NODQ-NEXT: divq %r10
+; NODQ-NEXT: movq %rax, %r10
; NODQ-NEXT: vpextrq $1, %xmm0, %rax
-; NODQ-NEXT: vpextrq $1, %xmm1, %rcx
+; NODQ-NEXT: vpextrq $1, %xmm1, %r11
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
+; NODQ-NEXT: divq %r11
+; NODQ-NEXT: vmovq %rcx, %xmm2
+; NODQ-NEXT: vmovq %rsi, %xmm3
+; NODQ-NEXT: vmovq %rdi, %xmm4
+; NODQ-NEXT: vmovq %r8, %xmm5
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm3[0],xmm2[0]
+; NODQ-NEXT: vmovq %r9, %xmm3
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm4 = xmm5[0],xmm4[0]
+; NODQ-NEXT: vinserti128 $1, %xmm2, %ymm4, %ymm2
+; NODQ-NEXT: vmovq %r10, %xmm4
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm4[0],xmm3[0]
; NODQ-NEXT: vmovq %rax, %xmm4
; NODQ-NEXT: vmovq %xmm0, %rax
; NODQ-NEXT: vmovq %xmm1, %rcx
@@ -376,44 +382,50 @@ define <8 x i64> @test_divv_8i64_narrow_strictfp(<8 x i64> %a, <8 x i64> %b) nou
; NODQ-NEXT: vpextrq $1, %xmm3, %rcx
; NODQ-NEXT: xorl %edx, %edx
; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm4
+; NODQ-NEXT: movq %rax, %rcx
; NODQ-NEXT: vmovq %xmm2, %rax
-; NODQ-NEXT: vmovq %xmm3, %rcx
+; NODQ-NEXT: vmovq %xmm3, %rsi
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm2
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm4[0]
-; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm3
-; NODQ-NEXT: vpextrq $1, %xmm3, %rax
-; NODQ-NEXT: vextracti32x4 $2, %zmm1, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rcx
+; NODQ-NEXT: divq %rsi
+; NODQ-NEXT: movq %rax, %rsi
+; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm2
+; NODQ-NEXT: vpextrq $1, %xmm2, %rax
+; NODQ-NEXT: vextracti32x4 $2, %zmm1, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %rdi
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm5
-; NODQ-NEXT: vmovq %xmm3, %rax
-; NODQ-NEXT: vmovq %xmm4, %rcx
+; NODQ-NEXT: divq %rdi
+; NODQ-NEXT: movq %rax, %rdi
+; NODQ-NEXT: vmovq %xmm2, %rax
+; NODQ-NEXT: vmovq %xmm3, %r8
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
-; NODQ-NEXT: vinserti128 $1, %xmm2, %ymm3, %ymm2
-; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm3
-; NODQ-NEXT: vpextrq $1, %xmm3, %rax
-; NODQ-NEXT: vextracti128 $1, %ymm1, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rcx
+; NODQ-NEXT: divq %r8
+; NODQ-NEXT: movq %rax, %r8
+; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm2
+; NODQ-NEXT: vpextrq $1, %xmm2, %rax
+; NODQ-NEXT: vextracti128 $1, %ymm1, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %r9
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm5
-; NODQ-NEXT: vmovq %xmm3, %rax
-; NODQ-NEXT: vmovq %xmm4, %rcx
+; NODQ-NEXT: divq %r9
+; NODQ-NEXT: movq %rax, %r9
+; NODQ-NEXT: vmovq %xmm2, %rax
+; NODQ-NEXT: vmovq %xmm3, %r10
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
+; NODQ-NEXT: divq %r10
+; NODQ-NEXT: movq %rax, %r10
; NODQ-NEXT: vpextrq $1, %xmm0, %rax
-; NODQ-NEXT: vpextrq $1, %xmm1, %rcx
+; NODQ-NEXT: vpextrq $1, %xmm1, %r11
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
+; NODQ-NEXT: divq %r11
+; NODQ-NEXT: vmovq %rcx, %xmm2
+; NODQ-NEXT: vmovq %rsi, %xmm3
+; NODQ-NEXT: vmovq %rdi, %xmm4
+; NODQ-NEXT: vmovq %r8, %xmm5
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm3[0],xmm2[0]
+; NODQ-NEXT: vmovq %r9, %xmm3
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm4 = xmm5[0],xmm4[0]
+; NODQ-NEXT: vinserti128 $1, %xmm2, %ymm4, %ymm2
+; NODQ-NEXT: vmovq %r10, %xmm4
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm4[0],xmm3[0]
; NODQ-NEXT: vmovq %rax, %xmm4
; NODQ-NEXT: vmovq %xmm0, %rax
; NODQ-NEXT: vmovq %xmm1, %rcx
@@ -449,39 +461,44 @@ define void @test_divv_7i64_narrow_strictfp(<7 x i64> %a, <7 x i64> %b, ptr %p)
; NODQ-NEXT: vpextrq $1, %xmm0, %rcx
; NODQ-NEXT: xorl %edx, %edx
; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm1
+; NODQ-NEXT: movq %rax, %rcx
; NODQ-NEXT: vmovq %xmm2, %rax
-; NODQ-NEXT: vmovq %xmm0, %rcx
+; NODQ-NEXT: vmovq %xmm0, %rsi
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm1 = xmm3[0],xmm1[0]
-; NODQ-NEXT: vextracti128 $1, %ymm2, %xmm3
-; NODQ-NEXT: vpextrq $1, %xmm3, %rax
-; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rcx
+; NODQ-NEXT: divq %rsi
+; NODQ-NEXT: movq %rax, %rsi
+; NODQ-NEXT: vextracti128 $1, %ymm2, %xmm1
+; NODQ-NEXT: vpextrq $1, %xmm1, %rax
+; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %r8
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm5
-; NODQ-NEXT: vmovq %xmm3, %rax
-; NODQ-NEXT: vmovq %xmm4, %rcx
+; NODQ-NEXT: divq %r8
+; NODQ-NEXT: movq %rax, %r8
+; NODQ-NEXT: vmovq %xmm1, %rax
+; NODQ-NEXT: vmovq %xmm3, %r9
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
-; NODQ-NEXT: vextracti32x4 $2, %zmm2, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rax
-; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm5
-; NODQ-NEXT: vpextrq $1, %xmm5, %rcx
+; NODQ-NEXT: divq %r9
+; NODQ-NEXT: movq %rax, %r9
+; NODQ-NEXT: vextracti32x4 $2, %zmm2, %xmm1
+; NODQ-NEXT: vpextrq $1, %xmm1, %rax
+; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %r10
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm6
-; NODQ-NEXT: vmovq %xmm4, %rax
-; NODQ-NEXT: vmovq %xmm5, %rcx
+; NODQ-NEXT: divq %r10
+; NODQ-NEXT: movq %rax, %r10
+; NODQ-NEXT: vmovq %xmm1, %rax
+; NODQ-NEXT: vmovq %xmm3, %r11
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm4
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm4 = xmm4[0],xmm6[0]
+; NODQ-NEXT: divq %r11
+; NODQ-NEXT: vmovq %rcx, %xmm1
+; NODQ-NEXT: vmovq %rsi, %xmm3
+; NODQ-NEXT: vmovq %r8, %xmm4
+; NODQ-NEXT: vmovq %r9, %xmm5
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm1 = xmm3[0],xmm1[0]
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm5[0],xmm4[0]
+; NODQ-NEXT: vmovq %r10, %xmm4
+; NODQ-NEXT: vmovq %rax, %xmm5
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm4 = xmm5[0],xmm4[0]
; NODQ-NEXT: vextracti32x4 $3, %zmm2, %xmm2
; NODQ-NEXT: vmovq %xmm2, %rax
; NODQ-NEXT: vextracti32x4 $3, %zmm0, %xmm0
diff --git a/llvm/test/CodeGen/X86/vector-idiv-udiv-512.ll b/llvm/test/CodeGen/X86/vector-idiv-udiv-512.ll
index 5fde70d7b083e..e3918f0bddf90 100644
--- a/llvm/test/CodeGen/X86/vector-idiv-udiv-512.ll
+++ b/llvm/test/CodeGen/X86/vector-idiv-udiv-512.ll
@@ -253,44 +253,50 @@ define <8 x i64> @test_divv_8i64(<8 x i64> %a, <8 x i64> %b) nounwind {
; NODQ-NEXT: vpextrq $1, %xmm3, %rcx
; NODQ-NEXT: xorl %edx, %edx
; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm4
+; NODQ-NEXT: movq %rax, %rcx
; NODQ-NEXT: vmovq %xmm2, %rax
-; NODQ-NEXT: vmovq %xmm3, %rcx
+; NODQ-NEXT: vmovq %xmm3, %rsi
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm2
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm4[0]
-; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm3
-; NODQ-NEXT: vpextrq $1, %xmm3, %rax
-; NODQ-NEXT: vextracti32x4 $2, %zmm1, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rcx
+; NODQ-NEXT: divq %rsi
+; NODQ-NEXT: movq %rax, %rsi
+; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm2
+; NODQ-NEXT: vpextrq $1, %xmm2, %rax
+; NODQ-NEXT: vextracti32x4 $2, %zmm1, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %rdi
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm5
-; NODQ-NEXT: vmovq %xmm3, %rax
-; NODQ-NEXT: vmovq %xmm4, %rcx
+; NODQ-NEXT: divq %rdi
+; NODQ-NEXT: movq %rax, %rdi
+; NODQ-NEXT: vmovq %xmm2, %rax
+; NODQ-NEXT: vmovq %xmm3, %r8
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
-; NODQ-NEXT: vinserti128 $1, %xmm2, %ymm3, %ymm2
-; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm3
-; NODQ-NEXT: vpextrq $1, %xmm3, %rax
-; NODQ-NEXT: vextracti128 $1, %ymm1, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rcx
+; NODQ-NEXT: divq %r8
+; NODQ-NEXT: movq %rax, %r8
+; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm2
+; NODQ-NEXT: vpextrq $1, %xmm2, %rax
+; NODQ-NEXT: vextracti128 $1, %ymm1, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %r9
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm5
-; NODQ-NEXT: vmovq %xmm3, %rax
-; NODQ-NEXT: vmovq %xmm4, %rcx
+; NODQ-NEXT: divq %r9
+; NODQ-NEXT: movq %rax, %r9
+; NODQ-NEXT: vmovq %xmm2, %rax
+; NODQ-NEXT: vmovq %xmm3, %r10
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
+; NODQ-NEXT: divq %r10
+; NODQ-NEXT: movq %rax, %r10
; NODQ-NEXT: vpextrq $1, %xmm0, %rax
-; NODQ-NEXT: vpextrq $1, %xmm1, %rcx
+; NODQ-NEXT: vpextrq $1, %xmm1, %r11
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
+; NODQ-NEXT: divq %r11
+; NODQ-NEXT: vmovq %rcx, %xmm2
+; NODQ-NEXT: vmovq %rsi, %xmm3
+; NODQ-NEXT: vmovq %rdi, %xmm4
+; NODQ-NEXT: vmovq %r8, %xmm5
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm3[0],xmm2[0]
+; NODQ-NEXT: vmovq %r9, %xmm3
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm4 = xmm5[0],xmm4[0]
+; NODQ-NEXT: vinserti128 $1, %xmm2, %ymm4, %ymm2
+; NODQ-NEXT: vmovq %r10, %xmm4
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm4[0],xmm3[0]
; NODQ-NEXT: vmovq %rax, %xmm4
; NODQ-NEXT: vmovq %xmm0, %rax
; NODQ-NEXT: vmovq %xmm1, %rcx
@@ -740,44 +746,50 @@ define <8 x i64> @test_divv_8i64_narrow(<8 x i64> %a, <8 x i64> %b) nounwind {
; NODQ-NEXT: vpextrq $1, %xmm3, %rcx
; NODQ-NEXT: xorl %edx, %edx
; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm4
+; NODQ-NEXT: movq %rax, %rcx
; NODQ-NEXT: vmovq %xmm2, %rax
-; NODQ-NEXT: vmovq %xmm3, %rcx
+; NODQ-NEXT: vmovq %xmm3, %rsi
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm2
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm4[0]
-; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm3
-; NODQ-NEXT: vpextrq $1, %xmm3, %rax
-; NODQ-NEXT: vextracti32x4 $2, %zmm1, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rcx
+; NODQ-NEXT: divq %rsi
+; NODQ-NEXT: movq %rax, %rsi
+; NODQ-NEXT: vextracti32x4 $2, %zmm0, %xmm2
+; NODQ-NEXT: vpextrq $1, %xmm2, %rax
+; NODQ-NEXT: vextracti32x4 $2, %zmm1, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %rdi
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm5
-; NODQ-NEXT: vmovq %xmm3, %rax
-; NODQ-NEXT: vmovq %xmm4, %rcx
+; NODQ-NEXT: divq %rdi
+; NODQ-NEXT: movq %rax, %rdi
+; NODQ-NEXT: vmovq %xmm2, %rax
+; NODQ-NEXT: vmovq %xmm3, %r8
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
-; NODQ-NEXT: vinserti128 $1, %xmm2, %ymm3, %ymm2
-; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm3
-; NODQ-NEXT: vpextrq $1, %xmm3, %rax
-; NODQ-NEXT: vextracti128 $1, %ymm1, %xmm4
-; NODQ-NEXT: vpextrq $1, %xmm4, %rcx
+; NODQ-NEXT: divq %r8
+; NODQ-NEXT: movq %rax, %r8
+; NODQ-NEXT: vextracti128 $1, %ymm0, %xmm2
+; NODQ-NEXT: vpextrq $1, %xmm2, %rax
+; NODQ-NEXT: vextracti128 $1, %ymm1, %xmm3
+; NODQ-NEXT: vpextrq $1, %xmm3, %r9
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm5
-; NODQ-NEXT: vmovq %xmm3, %rax
-; NODQ-NEXT: vmovq %xmm4, %rcx
+; NODQ-NEXT: divq %r9
+; NODQ-NEXT: movq %rax, %r9
+; NODQ-NEXT: vmovq %xmm2, %rax
+; NODQ-NEXT: vmovq %xmm3, %r10
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
-; NODQ-NEXT: vmovq %rax, %xmm3
-; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
+; NODQ-NEXT: divq %r10
+; NODQ-NEXT: movq %rax, %r10
; NODQ-NEXT: vpextrq $1, %xmm0, %rax
-; NODQ-NEXT: vpextrq $1, %xmm1, %rcx
+; NODQ-NEXT: vpextrq $1, %xmm1, %r11
; NODQ-NEXT: xorl %edx, %edx
-; NODQ-NEXT: divq %rcx
+; NODQ-NEXT: divq %r11
+; NODQ-NEXT: vmovq %rcx, %xmm2
+; NODQ-NEXT: vmovq %rsi, %xmm3
+; NODQ-NEXT: vmovq %rdi, %xmm4
+; NODQ-NEXT: vmovq %r8, %xmm5
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm2 = xmm3[0],xmm2[0]
+; NODQ-NEXT: vmovq %r9, %xmm3
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm4 = xmm5[0],xmm4[0]
+; NODQ-NEXT: vinserti128 $1, %xmm2, %ymm4, %ymm2
+; NODQ-NEXT: vmovq %r10, %xmm4
+; NODQ-NEXT: vpunpcklqdq {{.*#+}} xmm3 = xmm4[0],xmm3[0]
; NODQ-NEXT: vmovq %rax, %xmm4
; NODQ-NEXT: vmovq %xmm0, %rax
; NODQ-NEXT: vmovq %xmm1, %rcx
diff --git a/llvm/test/CodeGen/X86/xmulo.ll b/llvm/test/CodeGen/X86/xmulo.ll
index 3dee19784c461..d680ae11cea8a 100644
--- a/llvm/test/CodeGen/X86/xmulo.ll
+++ b/llvm/test/CodeGen/X86/xmulo.ll
@@ -90,9 +90,9 @@ define zeroext i1 @smuloi8(i8 %v1, i8 %v2, ptr %res) {
;
; WIN32-LABEL: smuloi8:
; WIN32: # %bb.0:
-; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: movzbl {{[0-9]+}}(%esp), %eax
; WIN32-NEXT: imulb {{[0-9]+}}(%esp)
+; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: setb %cl
; WIN32-NEXT: movb %al, (%edx)
; WIN32-NEXT: movl %ecx, %eax
@@ -303,9 +303,9 @@ define zeroext i1 @umuloi8(i8 %v1, i8 %v2, ptr %res) {
;
; WIN32-LABEL: umuloi8:
; WIN32: # %bb.0:
-; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: movzbl {{[0-9]+}}(%esp), %eax
; WIN32-NEXT: mulb {{[0-9]+}}(%esp)
+; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: setb %cl
; WIN32-NEXT: movb %al, (%edx)
; WIN32-NEXT: movl %ecx, %eax
@@ -1386,10 +1386,10 @@ define zeroext i1 @smuloi8_load(ptr %ptr1, i8 %v2, ptr %res) {
;
; WIN32-LABEL: smuloi8_load:
; WIN32: # %bb.0:
-; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: movl {{[0-9]+}}(%esp), %eax
; WIN32-NEXT: movzbl (%eax), %eax
; WIN32-NEXT: imulb {{[0-9]+}}(%esp)
+; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: setb %cl
; WIN32-NEXT: movb %al, (%edx)
; WIN32-NEXT: movl %ecx, %eax
@@ -1435,10 +1435,10 @@ define zeroext i1 @smuloi8_load2(i8 %v1, ptr %ptr2, ptr %res) {
;
; WIN32-LABEL: smuloi8_load2:
; WIN32: # %bb.0:
-; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: movzbl {{[0-9]+}}(%esp), %eax
; WIN32-NEXT: movl {{[0-9]+}}(%esp), %ecx
; WIN32-NEXT: imulb (%ecx)
+; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: setb %cl
; WIN32-NEXT: movb %al, (%edx)
; WIN32-NEXT: movl %ecx, %eax
@@ -1829,10 +1829,10 @@ define zeroext i1 @umuloi8_load(ptr %ptr1, i8 %v2, ptr %res) {
;
; WIN32-LABEL: umuloi8_load:
; WIN32: # %bb.0:
-; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: movl {{[0-9]+}}(%esp), %eax
; WIN32-NEXT: movzbl (%eax), %eax
; WIN32-NEXT: mulb {{[0-9]+}}(%esp)
+; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: setb %cl
; WIN32-NEXT: movb %al, (%edx)
; WIN32-NEXT: movl %ecx, %eax
@@ -1878,10 +1878,10 @@ define zeroext i1 @umuloi8_load2(i8 %v1, ptr %ptr2, ptr %res) {
;
; WIN32-LABEL: umuloi8_load2:
; WIN32: # %bb.0:
-; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: movzbl {{[0-9]+}}(%esp), %eax
; WIN32-NEXT: movl {{[0-9]+}}(%esp), %ecx
; WIN32-NEXT: mulb (%ecx)
+; WIN32-NEXT: movl {{[0-9]+}}(%esp), %edx
; WIN32-NEXT: setb %cl
; WIN32-NEXT: movb %al, (%edx)
; WIN32-NEXT: movl %ecx, %eax
More information about the llvm-commits
mailing list