[llvm] [X86] Fold `B + (-C)*A` into `neg + lea` (PR #215145)
Nikita Taranov via llvm-commits
llvm-commits at lists.llvm.org
Mon Aug 10 02:58:37 PDT 2026
https://github.com/nickitat updated https://github.com/llvm/llvm-project/pull/215145
>From 16320d39ba1a16fc7f4917bd7008b6ff2133eee9 Mon Sep 17 00:00:00 2001
From: Nikita Taranov <nickita.taranov at gmail.com>
Date: Sat, 8 Aug 2026 20:32:43 +0000
Subject: [PATCH 1/4] [X86] Form neg+lea for sub(X, shl(Y, C)) with C in [1,3]
(#37287)
x + -4*y currently compiles to
movl %edi, %eax; shll $2, %esi; subl %esi, %eax (3 uops, 7 bytes)
where GCC emits
negl %esi; leal (%rdi,%rsi,4), %eax (2 uops, 5 bytes)
Two independent canonicalizations converge on sub(X, shl(Y, C)): InstCombine
(from the multiply-by-negative-constant form) and the DAGCombiner fold
(add x, shl(0 - y, n)) -> sub(x, shl(y, n)) in visitADDLikeCommutative.
Nothing in the X86 backend breaks that apart again, so the scaled-index LEA is
never formed.
Worth noting where GCC's advantage actually comes from: it only reaches
neg+lea from the multiply spelling, not from an explicit x - (y << 2). GCC has
no equivalent of the (add x, shl(0 - y, n)) canonicalization, so the negate
survives expansion and its addressing-mode matcher picks the whole thing up.
The deficiency here is therefore the canonicalization rather than a missing
pattern, which is why this is fixed with an opt-out rather than a new combine.
(Inferred from GCC 13.3 output, not from reading its source.)
An X86-only combine cannot fix this by itself: it fights the generic fold and
DAGCombiner loops forever. So this adds a TargetLowering hook,
preferNegShlAddToSubShl(), which switches the generic fold off for exactly the
cases the X86 combine switches on, plus the reverse fold in combineSub. Loop
safety does not depend on that test being accurate, only on both sites asking
the same question: the two folds are exact complements by construction.
Both are restricted to AfterLegalizeDAG, and that phase gate is doing real
work rather than being a precaution. Inserting the NEG severs the shl from Y,
so any fold that would have reassociated shl(Y, C) into Y's own computation -
shl(mul(v, 15), 2) into mul(v, 60), or shl(and(srl(v, 2), 15), 3) into
and(shl(v, 1), 120), the shape bitfield extraction produces - is lost, and
those are usually the better deal. Letting the canonicalization run in the
earlier phases is what exposes Y to the shl so that reassociation can happen at
all; by the last phase it has had its chance, and what reaches this combine is
the shl that genuinely survived. An earlier version of this patch tried to
predict that with a profitability heuristic on Y's opcode instead, and got
mul-by-constant, add-by-constant and trunc/zext-wrapped bitfields wrong; all
three are now negative tests that pass without any such predicate.
This is mitigation rather than a guarantee: within the final phase, worklist
order still decides who reaches a node first, so a shl can in principle still
be taken before something else would have absorbed it. But it is right for the
reasons stated rather than by accident, and no case where it goes wrong turned
up in any of the measurements below.
Remaining profitability, in the order the guards appear:
- 64-bit mode only. The rewrite perturbs register allocation, and with six
allocatable registers in 32-bit mode that regularly costs a spill: over 200
modules of LLVM's own source, i386 came out net larger (29 modules worse
against 8 better) even though the fold itself was still shortening the
sequences it fired on.
- Y must die at the shl. NEG is two-address, so if Y is live afterwards (in
particular when Y is X, i.e. X - (X << C)) the negate needs its own MOV and
the sequence grows to three instructions. Without this the vector-idiv-*
and i128-udiv tests regress.
- X must not be a foldable load. The rewrite only pays off when the add turns
into an LEA; if X is a load it folds into the ADD as a memory operand
instead, the shl survives, and the NEG is added for nothing.
Measurements against trunk, with this version of the patch:
- llvm-test-suite SingleSource+MultiSource, 2106/2106 pass. Each benchmark
verifies its own output, so these are correctness results.
- 880 configurations of randomly generated programs - 220 seeds across
SSE2/AVX/AVX2/AVX512 - compiled by both compilers, executed, and checked
against each other and against clang -O2. No miscompiles.
- Size on real code is close to neutral. 240 modules of LLVM's own source,
8790 functions, 5.32MB of .text: 18 functions change, 14 smaller, 4 larger,
-104 bytes (-0.0020%). Best -36 bytes, worst +16; the larger cases are
stack-frame and spill-slot churn rather than the fold itself.
- check-llvm: 76802 tests, no failures.
Measurements carried over from the earlier version of the patch, which fired on
a different (not merely smaller) set of sites, so these describe the approach
rather than this exact code:
- A 150-shape synthetic sweep: 54 better, 0 worse, 96 unchanged, 702 -> 648
instructions, across 18 microarchitectures from nehalem to sapphirerapids,
znver1-5 and the Atom line (-0.08% to -0.29%). Counts instructions, not
uops, so it does not see the AMD scale cost noted below.
- No measurable effect on llvm-test-suite run time. Serialized alternating
runs put every benchmark inside the noise: the largest apparent regression
(miniFE, +12.7% over two passes) disappeared at seven passes, where that
benchmark's own baseline varies 16.8% run to run.
- A loop microbenchmark in the shape the fold targets - X still live across
the sub, so the baseline needs the extra MOV - runs 8.7% faster. Four other
kernels sit at the +-0.1% noise floor, measured against a control kernel
whose codegen the fold does not change. Host was Granite Rapids - which per
the paragraph below is one of the cores where the uop saving is nil, so this
number comes from the byte count rather than from uop pressure.
On the LEA, and on where this actually pays. The LEA usually has no
displacement and so is not a "3-operand LEA" for X86FixupLEAs, though
reassociation can fold a constant into one (see @add_const in the new test).
No subtarget splits it either way - checked across znver1-5, btver2, bdver2,
silvermont and atom, codegen is identical on all of them.
What varies is what the saved MOV is worth. Running llvm-mca over the 18
functions the fold changes in the corpus above, the uop delta is -22 on Haswell
and Skylake and on znver1/znver2, -23 on Jaguar, Bulldozer and Silvermont, but
only -6 on Zen3/4/5 and -1 on Sapphire Rapids. On the isolated shape the fold
targets it is -1 uop on the former group and exactly 0 on the latter. Two
different causes: Zen3 onwards charges an LEA with scale != 1 two uops, which
cancels the saved MOV, while Sapphire Rapids eliminates reg-reg MOVs at rename,
so there was nothing to save. With APX the SUB is not two-address at all - it
selects to "subq %rsi, %rdi, %rax" - and no MOV is needed to begin with.
So on the newest cores this is neutral rather than negative, and the size win
holds everywhere. That paragraph is model-based: no AMD hardware was
available.
So the case for this is matching GCC on the reported pattern and a real win in
the specific shape, not a general size or performance improvement.
New test covers both spellings, all three shift amounts, i32 and i64, and the
shapes where the fold must not fire. Four existing tests change (lea-opt,
cmp-select-sign, srem-vector-lkk, urem-vector-lkk), all smaller or neutral.
---
llvm/include/llvm/CodeGen/TargetLowering.h | 13 +
llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp | 8 +-
llvm/lib/Target/X86/X86ISelLowering.cpp | 65 +++
llvm/lib/Target/X86/X86ISelLowering.h | 2 +
llvm/test/CodeGen/X86/cmp-select-sign.ll | 134 +++---
llvm/test/CodeGen/X86/lea-opt.ll | 40 +-
llvm/test/CodeGen/X86/neg-shl-lea.ll | 443 ++++++++++++++++++
llvm/test/CodeGen/X86/srem-vector-lkk.ll | 24 +-
llvm/test/CodeGen/X86/urem-vector-lkk.ll | 12 +-
9 files changed, 631 insertions(+), 110 deletions(-)
create mode 100644 llvm/test/CodeGen/X86/neg-shl-lea.ll
diff --git a/llvm/include/llvm/CodeGen/TargetLowering.h b/llvm/include/llvm/CodeGen/TargetLowering.h
index 9a525d69b3ee8..c4ac7ec484f9b 100644
--- a/llvm/include/llvm/CodeGen/TargetLowering.h
+++ b/llvm/include/llvm/CodeGen/TargetLowering.h
@@ -964,6 +964,19 @@ class LLVM_ABI TargetLoweringBase {
return true;
}
+ /// Return true if the target prefers to keep (add x, (shl (0 - y), n)) over
+ /// the canonical (sub x, (shl y, n)). Targets with a scaled-index addressing
+ /// mode (e.g. x86 LEA) can fold the whole add+shl into one instruction, which
+ /// pays for the extra negate. \p ShAmt is the shift amount operand.
+ ///
+ /// Only consulted once the DAGCombiner reaches AfterLegalizeDAG: keeping the
+ /// negate in place stops (shl y, n) from being reassociated into y's own
+ /// computation, so the canonicalization has to run in the earlier phases for
+ /// that to get its chance first.
+ virtual bool preferNegShlAddToSubShl(EVT VT, SDValue ShAmt) const {
+ return false;
+ }
+
// By default prefer folding (abs (sub nsw x, y)) -> abds(x, y). Some targets
// may want to avoid this to prevent loss of sub_nsw pattern.
virtual bool preferABDSToABSWithNSW(EVT VT) const {
diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
index d7db5fb1b9e40..c053cac70c5b5 100644
--- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
@@ -3483,8 +3483,14 @@ SDValue DAGCombiner::visitADDLikeCommutative(SDValue N0, SDValue N1,
SDLoc DL(LocReference);
// fold (add x, shl(0 - y, n)) -> sub(x, shl(y, n))
+ // Skipped once the shift folds have settled if the target would rather keep
+ // the neg+shl+add form because it folds into a scaled-index address (e.g.
+ // x86 LEA). Only after AfterLegalizeDAG: running this fold in the earlier
+ // phases is what exposes y to the shl so it can be reassociated into y's own
+ // computation, which is usually the better deal.
SDValue Y, N;
- if (sd_match(N1, m_Shl(m_Neg(m_Value(Y)), m_Value(N))))
+ if (sd_match(N1, m_Shl(m_Neg(m_Value(Y)), m_Value(N))) &&
+ !(Level >= AfterLegalizeDAG && TLI.preferNegShlAddToSubShl(VT, N)))
return DAG.getNode(ISD::SUB, DL, VT, N0,
DAG.getNode(ISD::SHL, DL, VT, Y, N));
diff --git a/llvm/lib/Target/X86/X86ISelLowering.cpp b/llvm/lib/Target/X86/X86ISelLowering.cpp
index 3b39e1bddb8ba..be8370c9bf413 100644
--- a/llvm/lib/Target/X86/X86ISelLowering.cpp
+++ b/llvm/lib/Target/X86/X86ISelLowering.cpp
@@ -60227,6 +60227,39 @@ static SDValue combineSub(SDNode *N, SelectionDAG &DAG,
/*AllowOpaques*/ false);
};
+ // sub(X, shl(Y, C)) -> add(X, shl(0 - Y, C)) for C in [1,3], so that the
+ // add+shl folds into a single scaled-index LEA. This undoes the generic
+ // (add x, shl(0 - y, n)) -> (sub x, shl(y, n)) canonicalization, which is a
+ // pessimization here; preferNegShlAddToSubShl() turns that fold off for
+ // exactly these shift amounts so the two do not fight.
+ //
+ // Y must die at the shl: NEG is two-address, so if Y is live afterwards (in
+ // particular when Y is X itself, i.e. X - (X << C)) the negate needs its own
+ // MOV first, and the sequence grows to three instructions instead of two.
+ //
+ // X must not be a foldable load either. The rewrite only pays off when the
+ // add turns into an LEA (or folds into a memory operand); if X is a load it
+ // instead folds into the ADD as a memory operand, the shl survives, and the
+ // NEG is added for nothing - same instruction count, one more ALU op.
+ //
+ // Only after AfterLegalizeDAG. Inserting the NEG severs the shl from Y, so
+ // any fold that would have reassociated shl(Y, C) into Y's own computation -
+ // shl(mul(v, 15), 2) into mul(v, 60), or shl(and(srl(v, 2), 15), 3) into
+ // and(shl(v, 1), 120) - is lost, and those are usually the better deal.
+ // Running in the last combine phase means that reassociation has already had
+ // its chance, so what reaches us here is the shl that genuinely survived,
+ // rather than a guess about what the earlier phases were going to do.
+ if (DCI.isAfterLegalizeDAG() && !VT.isVector() &&
+ Op1.getOpcode() == ISD::SHL && Op1.hasOneUse() &&
+ Op1.getOperand(0).hasOneUse() && !isNullConstant(Op0) &&
+ !(Op0.getOpcode() == ISD::LOAD && Op0.hasOneUse()) &&
+ DAG.getTargetLoweringInfo().preferNegShlAddToSubShl(VT,
+ Op1.getOperand(1))) {
+ SDValue Neg = DAG.getNegative(Op1.getOperand(0), DL, VT);
+ SDValue Shl = DAG.getNode(ISD::SHL, DL, VT, Neg, Op1.getOperand(1));
+ return DAG.getNode(ISD::ADD, DL, VT, Op0, Shl);
+ }
+
// X86 can't encode an immediate LHS of a sub. See if we can push the
// negation into a preceding instruction. If the RHS of the sub is a XOR with
// one use and a constant, invert the immediate, saving one register.
@@ -63427,6 +63460,38 @@ SDValue X86TargetLowering::PerformDAGCombine(SDNode *N,
return SDValue();
}
+// (add x, (shl (0 - y), n)) selects to NEG + a single scaled-index LEA when n
+// is 1, 2 or 3. The canonical (sub x, (shl y, n)) needs SHL + SUB, plus a MOV
+// when x is still live because SUB is two-address - so this saves an
+// instruction and two bytes exactly when that MOV is needed, and breaks even
+// otherwise.
+//
+// The LEA normally has no displacement and so is not a "3-operand LEA" for
+// X86FixupLEAs, though reassociation can fold a constant into one (see
+// @add_const in neg-shl-lea.ll); no subtarget splits it either way.
+//
+// What the saved MOV is worth varies more than the LEA does. By LLVM's
+// scheduling models it is a uop on Haswell/Skylake-class Intel and on AMD up
+// to Zen2, plus Jaguar, Bulldozer and Silvermont. It is nothing on Zen3 and
+// later, which charge an LEA with scale != 1 two uops, and nothing on Sapphire
+// Rapids, which eliminates reg-reg MOVs at rename. With APX the SUB is not
+// two-address to begin with and no MOV is needed at all. So on the newest
+// cores this is neutral rather than negative, and the size win holds
+// everywhere.
+bool X86TargetLowering::preferNegShlAddToSubShl(EVT VT, SDValue ShAmt) const {
+ // 64-bit mode only. The rewrite is a small win on instruction count but it
+ // perturbs register allocation, and with only six allocatable registers in
+ // 32-bit mode that regularly costs a spill: over 200 modules of LLVM's own
+ // source, i386 came out net larger (29 modules worse against 8 better) even
+ // though the fold itself was still shortening the sequences it fired on.
+ if (!Subtarget.is64Bit())
+ return false;
+ if (VT != MVT::i32 && VT != MVT::i64)
+ return false;
+ auto *C = dyn_cast<ConstantSDNode>(ShAmt);
+ return C && C->getZExtValue() >= 1 && C->getZExtValue() <= 3;
+}
+
bool X86TargetLowering::preferABDSToABSWithNSW(EVT VT) const {
return Subtarget.canUseCMOV() && (VT == MVT::i32 || VT == MVT::i64);
}
diff --git a/llvm/lib/Target/X86/X86ISelLowering.h b/llvm/lib/Target/X86/X86ISelLowering.h
index a4b4e6f32591b..fdb3a01968017 100644
--- a/llvm/lib/Target/X86/X86ISelLowering.h
+++ b/llvm/lib/Target/X86/X86ISelLowering.h
@@ -184,6 +184,8 @@ namespace llvm {
SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
+ bool preferNegShlAddToSubShl(EVT VT, SDValue ShAmt) const override;
+
bool preferABDSToABSWithNSW(EVT VT) const override;
bool preferSextInRegOfTruncate(EVT TruncVT, EVT VT,
diff --git a/llvm/test/CodeGen/X86/cmp-select-sign.ll b/llvm/test/CodeGen/X86/cmp-select-sign.ll
index be6f2a6d05192..a382b73fa1262 100644
--- a/llvm/test/CodeGen/X86/cmp-select-sign.ll
+++ b/llvm/test/CodeGen/X86/cmp-select-sign.ll
@@ -576,43 +576,40 @@ define <4 x i65> @sign_4xi65(<4 x i65> %a) {
; CHECK-NOBMI-NEXT: movq %rdi, %rax
; CHECK-NOBMI-NEXT: movq {{[0-9]+}}(%rsp), %rcx
; CHECK-NOBMI-NEXT: andl $1, %ecx
-; CHECK-NOBMI-NEXT: movq %rcx, %rsi
+; CHECK-NOBMI-NEXT: negq %rcx
+; CHECK-NOBMI-NEXT: movq {{[0-9]+}}(%rsp), %rsi
+; CHECK-NOBMI-NEXT: andl $1, %esi
; CHECK-NOBMI-NEXT: negq %rsi
-; CHECK-NOBMI-NEXT: movq {{[0-9]+}}(%rsp), %rdi
-; CHECK-NOBMI-NEXT: andl $1, %edi
-; CHECK-NOBMI-NEXT: movq %rdi, %r10
-; CHECK-NOBMI-NEXT: negq %r10
; CHECK-NOBMI-NEXT: andl $1, %edx
; CHECK-NOBMI-NEXT: negq %rdx
; CHECK-NOBMI-NEXT: andl $1, %r8d
; CHECK-NOBMI-NEXT: negq %r8
-; CHECK-NOBMI-NEXT: leaq (%r8,%r8), %r9
+; CHECK-NOBMI-NEXT: leaq (%r8,%r8), %rdi
; CHECK-NOBMI-NEXT: movq %rdx, %xmm0
; CHECK-NOBMI-NEXT: orq $1, %rdx
; CHECK-NOBMI-NEXT: movq %rdx, (%rax)
-; CHECK-NOBMI-NEXT: andl $15, %r10d
-; CHECK-NOBMI-NEXT: movb %r10b, 32(%rax)
-; CHECK-NOBMI-NEXT: movl %esi, %r10d
-; CHECK-NOBMI-NEXT: andl $1, %r10d
-; CHECK-NOBMI-NEXT: shldq $2, %rsi, %r10
-; CHECK-NOBMI-NEXT: shll $3, %edi
-; CHECK-NOBMI-NEXT: subq %rdi, %r10
-; CHECK-NOBMI-NEXT: orq $8, %r10
-; CHECK-NOBMI-NEXT: movq %r10, 24(%rax)
+; CHECK-NOBMI-NEXT: movl %ecx, %r9d
+; CHECK-NOBMI-NEXT: andl $1, %r9d
+; CHECK-NOBMI-NEXT: shldq $2, %rcx, %r9
+; CHECK-NOBMI-NEXT: leaq (%r9,%rsi,8), %r9
+; CHECK-NOBMI-NEXT: # kill: def $esi killed $esi killed $rsi
+; CHECK-NOBMI-NEXT: andl $15, %esi
+; CHECK-NOBMI-NEXT: movb %sil, 32(%rax)
+; CHECK-NOBMI-NEXT: orq $8, %r9
+; CHECK-NOBMI-NEXT: movq %r9, 24(%rax)
; CHECK-NOBMI-NEXT: movl %r8d, %esi
; CHECK-NOBMI-NEXT: andl $1, %esi
; CHECK-NOBMI-NEXT: shldq $1, %r8, %rsi
-; CHECK-NOBMI-NEXT: shll $2, %ecx
-; CHECK-NOBMI-NEXT: subq %rcx, %rsi
-; CHECK-NOBMI-NEXT: orq $4, %rsi
-; CHECK-NOBMI-NEXT: movq %rsi, 16(%rax)
+; CHECK-NOBMI-NEXT: leaq (%rsi,%rcx,4), %rcx
+; CHECK-NOBMI-NEXT: orq $4, %rcx
+; CHECK-NOBMI-NEXT: movq %rcx, 16(%rax)
; CHECK-NOBMI-NEXT: movq %rdx, %xmm1
; CHECK-NOBMI-NEXT: punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]
; CHECK-NOBMI-NEXT: pshufd {{.*#+}} xmm0 = xmm1[2,3,2,3]
; CHECK-NOBMI-NEXT: movq %xmm0, %rcx
-; CHECK-NOBMI-NEXT: orq $2, %r9
-; CHECK-NOBMI-NEXT: subq %rcx, %r9
-; CHECK-NOBMI-NEXT: movq %r9, 8(%rax)
+; CHECK-NOBMI-NEXT: orq $2, %rdi
+; CHECK-NOBMI-NEXT: subq %rcx, %rdi
+; CHECK-NOBMI-NEXT: movq %rdi, 8(%rax)
; CHECK-NOBMI-NEXT: retq
;
; CHECK-BMI2-SSE2-LABEL: sign_4xi65:
@@ -620,84 +617,75 @@ define <4 x i65> @sign_4xi65(<4 x i65> %a) {
; CHECK-BMI2-SSE2-NEXT: movq %rdi, %rax
; CHECK-BMI2-SSE2-NEXT: movq {{[0-9]+}}(%rsp), %rcx
; CHECK-BMI2-SSE2-NEXT: andl $1, %ecx
-; CHECK-BMI2-SSE2-NEXT: movq %rcx, %rsi
+; CHECK-BMI2-SSE2-NEXT: negq %rcx
+; CHECK-BMI2-SSE2-NEXT: movq {{[0-9]+}}(%rsp), %rsi
+; CHECK-BMI2-SSE2-NEXT: andl $1, %esi
; CHECK-BMI2-SSE2-NEXT: negq %rsi
-; CHECK-BMI2-SSE2-NEXT: movq {{[0-9]+}}(%rsp), %rdi
-; CHECK-BMI2-SSE2-NEXT: andl $1, %edi
-; CHECK-BMI2-SSE2-NEXT: movq %rdi, %r10
-; CHECK-BMI2-SSE2-NEXT: negq %r10
; CHECK-BMI2-SSE2-NEXT: andl $1, %edx
; CHECK-BMI2-SSE2-NEXT: negq %rdx
; CHECK-BMI2-SSE2-NEXT: andl $1, %r8d
; CHECK-BMI2-SSE2-NEXT: negq %r8
-; CHECK-BMI2-SSE2-NEXT: leaq (%r8,%r8), %r9
+; CHECK-BMI2-SSE2-NEXT: leaq (%r8,%r8), %rdi
; CHECK-BMI2-SSE2-NEXT: movq %rdx, %xmm0
; CHECK-BMI2-SSE2-NEXT: orq $1, %rdx
; CHECK-BMI2-SSE2-NEXT: movq %rdx, (%rax)
-; CHECK-BMI2-SSE2-NEXT: andl $15, %r10d
-; CHECK-BMI2-SSE2-NEXT: movb %r10b, 32(%rax)
-; CHECK-BMI2-SSE2-NEXT: movl %esi, %r10d
-; CHECK-BMI2-SSE2-NEXT: andl $1, %r10d
-; CHECK-BMI2-SSE2-NEXT: shldq $2, %rsi, %r10
-; CHECK-BMI2-SSE2-NEXT: shll $3, %edi
-; CHECK-BMI2-SSE2-NEXT: subq %rdi, %r10
-; CHECK-BMI2-SSE2-NEXT: orq $8, %r10
-; CHECK-BMI2-SSE2-NEXT: movq %r10, 24(%rax)
+; CHECK-BMI2-SSE2-NEXT: movl %ecx, %r9d
+; CHECK-BMI2-SSE2-NEXT: andl $1, %r9d
+; CHECK-BMI2-SSE2-NEXT: shldq $2, %rcx, %r9
+; CHECK-BMI2-SSE2-NEXT: leaq (%r9,%rsi,8), %r9
+; CHECK-BMI2-SSE2-NEXT: # kill: def $esi killed $esi killed $rsi
+; CHECK-BMI2-SSE2-NEXT: andl $15, %esi
+; CHECK-BMI2-SSE2-NEXT: movb %sil, 32(%rax)
+; CHECK-BMI2-SSE2-NEXT: orq $8, %r9
+; CHECK-BMI2-SSE2-NEXT: movq %r9, 24(%rax)
; CHECK-BMI2-SSE2-NEXT: movl %r8d, %esi
; CHECK-BMI2-SSE2-NEXT: andl $1, %esi
; CHECK-BMI2-SSE2-NEXT: shldq $1, %r8, %rsi
-; CHECK-BMI2-SSE2-NEXT: shll $2, %ecx
-; CHECK-BMI2-SSE2-NEXT: subq %rcx, %rsi
-; CHECK-BMI2-SSE2-NEXT: orq $4, %rsi
-; CHECK-BMI2-SSE2-NEXT: movq %rsi, 16(%rax)
+; CHECK-BMI2-SSE2-NEXT: leaq (%rsi,%rcx,4), %rcx
+; CHECK-BMI2-SSE2-NEXT: orq $4, %rcx
+; CHECK-BMI2-SSE2-NEXT: movq %rcx, 16(%rax)
; CHECK-BMI2-SSE2-NEXT: movq %rdx, %xmm1
; CHECK-BMI2-SSE2-NEXT: punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]
; CHECK-BMI2-SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm1[2,3,2,3]
; CHECK-BMI2-SSE2-NEXT: movq %xmm0, %rcx
-; CHECK-BMI2-SSE2-NEXT: orq $2, %r9
-; CHECK-BMI2-SSE2-NEXT: subq %rcx, %r9
-; CHECK-BMI2-SSE2-NEXT: movq %r9, 8(%rax)
+; CHECK-BMI2-SSE2-NEXT: orq $2, %rdi
+; CHECK-BMI2-SSE2-NEXT: subq %rcx, %rdi
+; CHECK-BMI2-SSE2-NEXT: movq %rdi, 8(%rax)
; CHECK-BMI2-SSE2-NEXT: retq
;
; CHECK-AVX12-LABEL: sign_4xi65:
; CHECK-AVX12: # %bb.0:
; CHECK-AVX12-NEXT: movq %rdi, %rax
-; CHECK-AVX12-NEXT: andl $1, %r8d
-; CHECK-AVX12-NEXT: movq %r8, %rsi
-; CHECK-AVX12-NEXT: negq %rsi
; CHECK-AVX12-NEXT: movq {{[0-9]+}}(%rsp), %rcx
; CHECK-AVX12-NEXT: andl $1, %ecx
-; CHECK-AVX12-NEXT: movq %rcx, %r9
-; CHECK-AVX12-NEXT: negq %r9
-; CHECK-AVX12-NEXT: movq {{[0-9]+}}(%rsp), %rdi
-; CHECK-AVX12-NEXT: andl $1, %edi
-; CHECK-AVX12-NEXT: movq %rdi, %r10
-; CHECK-AVX12-NEXT: negq %r10
+; CHECK-AVX12-NEXT: negq %rcx
+; CHECK-AVX12-NEXT: movq {{[0-9]+}}(%rsp), %rsi
+; CHECK-AVX12-NEXT: andl $1, %esi
+; CHECK-AVX12-NEXT: negq %rsi
+; CHECK-AVX12-NEXT: andl $1, %r8d
+; CHECK-AVX12-NEXT: negq %r8
; CHECK-AVX12-NEXT: andl $1, %edx
-; CHECK-AVX12-NEXT: movq %rdx, %r11
-; CHECK-AVX12-NEXT: negq %r11
-; CHECK-AVX12-NEXT: orq $1, %r11
-; CHECK-AVX12-NEXT: movq %r11, (%rax)
-; CHECK-AVX12-NEXT: andl $15, %r10d
-; CHECK-AVX12-NEXT: movb %r10b, 32(%rax)
-; CHECK-AVX12-NEXT: addl %r8d, %r8d
-; CHECK-AVX12-NEXT: subq %r8, %rdx
-; CHECK-AVX12-NEXT: orq $2, %rdx
-; CHECK-AVX12-NEXT: movq %rdx, 8(%rax)
-; CHECK-AVX12-NEXT: movl %r9d, %edx
+; CHECK-AVX12-NEXT: leaq (%rdx,%r8,2), %rdi
+; CHECK-AVX12-NEXT: negq %rdx
+; CHECK-AVX12-NEXT: orq $1, %rdx
+; CHECK-AVX12-NEXT: movq %rdx, (%rax)
+; CHECK-AVX12-NEXT: orq $2, %rdi
+; CHECK-AVX12-NEXT: movq %rdi, 8(%rax)
+; CHECK-AVX12-NEXT: movl %ecx, %edx
; CHECK-AVX12-NEXT: andl $1, %edx
-; CHECK-AVX12-NEXT: shldq $2, %r9, %rdx
-; CHECK-AVX12-NEXT: shll $3, %edi
-; CHECK-AVX12-NEXT: subq %rdi, %rdx
+; CHECK-AVX12-NEXT: shldq $2, %rcx, %rdx
+; CHECK-AVX12-NEXT: leaq (%rdx,%rsi,8), %rdx
+; CHECK-AVX12-NEXT: # kill: def $esi killed $esi killed $rsi
+; CHECK-AVX12-NEXT: andl $15, %esi
+; CHECK-AVX12-NEXT: movb %sil, 32(%rax)
; CHECK-AVX12-NEXT: orq $8, %rdx
; CHECK-AVX12-NEXT: movq %rdx, 24(%rax)
-; CHECK-AVX12-NEXT: movl %esi, %edx
+; CHECK-AVX12-NEXT: movl %r8d, %edx
; CHECK-AVX12-NEXT: andl $1, %edx
-; CHECK-AVX12-NEXT: shldq $1, %rsi, %rdx
-; CHECK-AVX12-NEXT: shll $2, %ecx
-; CHECK-AVX12-NEXT: subq %rcx, %rdx
-; CHECK-AVX12-NEXT: orq $4, %rdx
-; CHECK-AVX12-NEXT: movq %rdx, 16(%rax)
+; CHECK-AVX12-NEXT: shldq $1, %r8, %rdx
+; CHECK-AVX12-NEXT: leaq (%rdx,%rcx,4), %rcx
+; CHECK-AVX12-NEXT: orq $4, %rcx
+; CHECK-AVX12-NEXT: movq %rcx, 16(%rax)
; CHECK-AVX12-NEXT: retq
;
; CHECK-AVX512-LABEL: sign_4xi65:
diff --git a/llvm/test/CodeGen/X86/lea-opt.ll b/llvm/test/CodeGen/X86/lea-opt.ll
index 88712328e54a7..58747d99048ff 100644
--- a/llvm/test/CodeGen/X86/lea-opt.ll
+++ b/llvm/test/CodeGen/X86/lea-opt.ll
@@ -311,9 +311,10 @@ sw.epilog: ; preds = %sw.bb.2, %sw.bb.1,
define i32 @test5(i32 %x, i32 %y) #0 {
; CHECK-LABEL: test5:
; CHECK: # %bb.0: # %entry
-; CHECK-NEXT: movl %edi, %eax
-; CHECK-NEXT: addl %esi, %esi
-; CHECK-NEXT: subl %esi, %eax
+; CHECK-NEXT: # kill: def $esi killed $esi def $rsi
+; CHECK-NEXT: # kill: def $edi killed $edi def $rdi
+; CHECK-NEXT: negl %esi
+; CHECK-NEXT: leal (%rdi,%rsi,2), %eax
; CHECK-NEXT: retq
entry:
%mul = mul nsw i32 %y, -2
@@ -338,9 +339,10 @@ entry:
define i32 @test7(i32 %x, i32 %y) #0 {
; CHECK-LABEL: test7:
; CHECK: # %bb.0: # %entry
-; CHECK-NEXT: movl %edi, %eax
-; CHECK-NEXT: shll $2, %esi
-; CHECK-NEXT: subl %esi, %eax
+; CHECK-NEXT: # kill: def $esi killed $esi def $rsi
+; CHECK-NEXT: # kill: def $edi killed $edi def $rdi
+; CHECK-NEXT: negl %esi
+; CHECK-NEXT: leal (%rdi,%rsi,4), %eax
; CHECK-NEXT: retq
entry:
%mul = mul nsw i32 %y, -4
@@ -365,9 +367,10 @@ entry:
define i32 @test9(i32 %x, i32 %y) #0 {
; CHECK-LABEL: test9:
; CHECK: # %bb.0: # %entry
-; CHECK-NEXT: movl %edi, %eax
-; CHECK-NEXT: addl %esi, %esi
-; CHECK-NEXT: subl %esi, %eax
+; CHECK-NEXT: # kill: def $esi killed $esi def $rsi
+; CHECK-NEXT: # kill: def $edi killed $edi def $rdi
+; CHECK-NEXT: negl %esi
+; CHECK-NEXT: leal (%rdi,%rsi,2), %eax
; CHECK-NEXT: retq
entry:
%mul = mul nsw i32 -2, %y
@@ -392,9 +395,10 @@ entry:
define i32 @test11(i32 %x, i32 %y) #0 {
; CHECK-LABEL: test11:
; CHECK: # %bb.0: # %entry
-; CHECK-NEXT: movl %edi, %eax
-; CHECK-NEXT: shll $2, %esi
-; CHECK-NEXT: subl %esi, %eax
+; CHECK-NEXT: # kill: def $esi killed $esi def $rsi
+; CHECK-NEXT: # kill: def $edi killed $edi def $rdi
+; CHECK-NEXT: negl %esi
+; CHECK-NEXT: leal (%rdi,%rsi,4), %eax
; CHECK-NEXT: retq
entry:
%mul = mul nsw i32 -4, %y
@@ -418,9 +422,8 @@ entry:
define i64 @test13(i64 %x, i64 %y) #0 {
; CHECK-LABEL: test13:
; CHECK: # %bb.0: # %entry
-; CHECK-NEXT: movq %rdi, %rax
-; CHECK-NEXT: shlq $2, %rsi
-; CHECK-NEXT: subq %rsi, %rax
+; CHECK-NEXT: negq %rsi
+; CHECK-NEXT: leaq (%rdi,%rsi,4), %rax
; CHECK-NEXT: retq
entry:
%mul = mul nsw i64 -4, %y
@@ -444,9 +447,10 @@ entry:
define zeroext i16 @test15(i16 zeroext %x, i16 zeroext %y) #0 {
; CHECK-LABEL: test15:
; CHECK: # %bb.0: # %entry
-; CHECK-NEXT: movl %edi, %eax
-; CHECK-NEXT: shll $3, %esi
-; CHECK-NEXT: subl %esi, %eax
+; CHECK-NEXT: # kill: def $esi killed $esi def $rsi
+; CHECK-NEXT: # kill: def $edi killed $edi def $rdi
+; CHECK-NEXT: negl %esi
+; CHECK-NEXT: leal (%rdi,%rsi,8), %eax
; CHECK-NEXT: # kill: def $ax killed $ax killed $eax
; CHECK-NEXT: retq
entry:
diff --git a/llvm/test/CodeGen/X86/neg-shl-lea.ll b/llvm/test/CodeGen/X86/neg-shl-lea.ll
new file mode 100644
index 0000000000000..3f5878595bb12
--- /dev/null
+++ b/llvm/test/CodeGen/X86/neg-shl-lea.ll
@@ -0,0 +1,443 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 6
+; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --check-prefix=X64
+; RUN: llc < %s -mtriple=i686-unknown-linux-gnu | FileCheck %s --check-prefix=X86
+
+; sub(X, shl(Y, C)) with C in [1,3] becomes neg + a scaled-index LEA, saving
+; the MOV that the destructive SUB would otherwise need. See issue #37287.
+;
+; The fold is 64-bit mode only, so every X86 check below is a negative one.
+; The guard tests use i32 to keep the 32-bit output readable; i64 in 32-bit
+; mode is split into register pairs and says nothing extra.
+
+define i64 @shl1(i64 %x, i64 %y) {
+; X64-LABEL: shl1:
+; X64: # %bb.0:
+; X64-NEXT: negq %rsi
+; X64-NEXT: leaq (%rdi,%rsi,2), %rax
+; X64-NEXT: retq
+;
+; X86-LABEL: shl1:
+; X86: # %bb.0:
+; X86-NEXT: pushl %esi
+; X86-NEXT: .cfi_def_cfa_offset 8
+; X86-NEXT: .cfi_offset %esi, -8
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
+; X86-NEXT: shldl $1, %ecx, %esi
+; X86-NEXT: addl %ecx, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: sbbl %esi, %edx
+; X86-NEXT: popl %esi
+; X86-NEXT: .cfi_def_cfa_offset 4
+; X86-NEXT: retl
+ %shl = shl i64 %y, 1
+ %r = sub i64 %x, %shl
+ ret i64 %r
+}
+
+define i64 @shl2(i64 %x, i64 %y) {
+; X64-LABEL: shl2:
+; X64: # %bb.0:
+; X64-NEXT: negq %rsi
+; X64-NEXT: leaq (%rdi,%rsi,4), %rax
+; X64-NEXT: retq
+;
+; X86-LABEL: shl2:
+; X86: # %bb.0:
+; X86-NEXT: pushl %esi
+; X86-NEXT: .cfi_def_cfa_offset 8
+; X86-NEXT: .cfi_offset %esi, -8
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
+; X86-NEXT: shldl $2, %ecx, %esi
+; X86-NEXT: shll $2, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: sbbl %esi, %edx
+; X86-NEXT: popl %esi
+; X86-NEXT: .cfi_def_cfa_offset 4
+; X86-NEXT: retl
+ %shl = shl i64 %y, 2
+ %r = sub i64 %x, %shl
+ ret i64 %r
+}
+
+define i64 @shl3(i64 %x, i64 %y) {
+; X64-LABEL: shl3:
+; X64: # %bb.0:
+; X64-NEXT: negq %rsi
+; X64-NEXT: leaq (%rdi,%rsi,8), %rax
+; X64-NEXT: retq
+;
+; X86-LABEL: shl3:
+; X86: # %bb.0:
+; X86-NEXT: pushl %esi
+; X86-NEXT: .cfi_def_cfa_offset 8
+; X86-NEXT: .cfi_offset %esi, -8
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
+; X86-NEXT: shldl $3, %ecx, %esi
+; X86-NEXT: shll $3, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: sbbl %esi, %edx
+; X86-NEXT: popl %esi
+; X86-NEXT: .cfi_def_cfa_offset 4
+; X86-NEXT: retl
+ %shl = shl i64 %y, 3
+ %r = sub i64 %x, %shl
+ ret i64 %r
+}
+
+define i32 @shl2_i32(i32 %x, i32 %y) {
+; X64-LABEL: shl2_i32:
+; X64: # %bb.0:
+; X64-NEXT: # kill: def $esi killed $esi def $rsi
+; X64-NEXT: # kill: def $edi killed $edi def $rdi
+; X64-NEXT: negl %esi
+; X64-NEXT: leal (%rdi,%rsi,4), %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: shl2_i32:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: shll $2, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ %shl = shl i32 %y, 2
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; The multiply spelling from the original report: x + -4*y.
+define i32 @mul_form(i32 %x, i32 %y) {
+; X64-LABEL: mul_form:
+; X64: # %bb.0:
+; X64-NEXT: # kill: def $esi killed $esi def $rsi
+; X64-NEXT: # kill: def $edi killed $edi def $rdi
+; X64-NEXT: negl %esi
+; X64-NEXT: leal (%rdi,%rsi,4), %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: mul_form:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: shll $2, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ %m = mul i32 %y, -4
+ %r = add i32 %x, %m
+ ret i32 %r
+}
+
+; The other canonical spelling, add(x, shl(0 - y, n)). This one reaches the
+; fold through the TargetLowering hook rather than through combineSub, so it
+; covers the second call site.
+define i32 @add_neg_form(i32 %x, i32 %y) {
+; X64-LABEL: add_neg_form:
+; X64: # %bb.0:
+; X64-NEXT: # kill: def $esi killed $esi def $rsi
+; X64-NEXT: # kill: def $edi killed $edi def $rdi
+; X64-NEXT: negl %esi
+; X64-NEXT: leal (%rdi,%rsi,8), %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: add_neg_form:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: shll $3, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ %n = sub i32 0, %y
+ %shl = shl i32 %n, 3
+ %r = add i32 %x, %shl
+ ret i32 %r
+}
+
+; X is still live afterwards, so the baseline needs an extra MOV and the
+; rewrite is at its most profitable.
+define i64 @x_live_after(i64 %x, i64 %y) {
+; X64-LABEL: x_live_after:
+; X64: # %bb.0:
+; X64-NEXT: negq %rsi
+; X64-NEXT: leaq (%rdi,%rsi,8), %rax
+; X64-NEXT: xorq %rdi, %rax
+; X64-NEXT: retq
+;
+; X86-LABEL: x_live_after:
+; X86: # %bb.0:
+; X86-NEXT: pushl %edi
+; X86-NEXT: .cfi_def_cfa_offset 8
+; X86-NEXT: pushl %esi
+; X86-NEXT: .cfi_def_cfa_offset 12
+; X86-NEXT: .cfi_offset %esi, -12
+; X86-NEXT: .cfi_offset %edi, -8
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
+; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT: shldl $3, %edx, %edi
+; X86-NEXT: shll $3, %edx
+; X86-NEXT: movl %ecx, %eax
+; X86-NEXT: subl %edx, %eax
+; X86-NEXT: movl %esi, %edx
+; X86-NEXT: sbbl %edi, %edx
+; X86-NEXT: xorl %ecx, %eax
+; X86-NEXT: xorl %esi, %edx
+; X86-NEXT: popl %esi
+; X86-NEXT: .cfi_def_cfa_offset 8
+; X86-NEXT: popl %edi
+; X86-NEXT: .cfi_def_cfa_offset 4
+; X86-NEXT: retl
+ %shl = shl i64 %y, 3
+ %s = sub i64 %x, %shl
+ %r = xor i64 %s, %x
+ ret i64 %r
+}
+
+;; Negative tests.
+
+; C = 0 and C >= 4 are outside the LEA scale range.
+define i32 @shl0(i32 %x, i32 %y) {
+; X64-LABEL: shl0:
+; X64: # %bb.0:
+; X64-NEXT: movl %edi, %eax
+; X64-NEXT: subl %esi, %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: shl0:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: subl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: retl
+ %shl = shl i32 %y, 0
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+define i32 @shl4(i32 %x, i32 %y) {
+; X64-LABEL: shl4:
+; X64: # %bb.0:
+; X64-NEXT: movl %edi, %eax
+; X64-NEXT: shll $4, %esi
+; X64-NEXT: subl %esi, %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: shl4:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: shll $4, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ %shl = shl i32 %y, 4
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; Y survives the shift, so NEG - which is two-address - would need its own MOV
+; and the sequence would grow rather than shrink.
+define i32 @y_multi_use(i32 %x, i32 %y, ptr %p) {
+; X64-LABEL: y_multi_use:
+; X64: # %bb.0:
+; X64-NEXT: movl %edi, %eax
+; X64-NEXT: movl %esi, (%rdx)
+; X64-NEXT: shll $2, %esi
+; X64-NEXT: subl %esi, %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: y_multi_use:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT: movl %ecx, (%edx)
+; X86-NEXT: shll $2, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ store i32 %y, ptr %p
+ %shl = shl i32 %y, 2
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; Y is X itself, which is the same problem.
+define i32 @x_minus_x_shl(i32 %x) {
+; X64-LABEL: x_minus_x_shl:
+; X64: # %bb.0:
+; X64-NEXT: movl %edi, %eax
+; X64-NEXT: leal (,%rax,8), %ecx
+; X64-NEXT: subl %ecx, %eax
+; X64-NEXT: # kill: def $eax killed $eax killed $rax
+; X64-NEXT: retq
+;
+; X86-LABEL: x_minus_x_shl:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: leal (,%eax,8), %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ %shl = shl i32 %x, 3
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; X is a foldable load, so it becomes a memory operand of the ADD, the shift
+; survives, and the NEG would be added for nothing.
+define i32 @x_is_load(ptr %p, i32 %y) {
+; X64-LABEL: x_is_load:
+; X64: # %bb.0:
+; X64-NEXT: movl (%rdi), %eax
+; X64-NEXT: shll $2, %esi
+; X64-NEXT: subl %esi, %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: x_is_load:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl (%eax), %eax
+; X86-NEXT: shll $2, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ %x = load i32, ptr %p
+ %shl = shl i32 %y, 2
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; The shift is absorbed into Y's own computation before this combine runs:
+; shl(and(srl(c, 2), 15), 3) becomes and(shl(c, 1), 120), which is free. By
+; AfterLegalizeDAG there is no shl left to turn into an LEA scale.
+define i32 @bitfield(i32 %x, i32 %c) {
+; X64-LABEL: bitfield:
+; X64: # %bb.0:
+; X64-NEXT: movl %edi, %eax
+; X64-NEXT: addl %esi, %esi
+; X64-NEXT: andl $120, %esi
+; X64-NEXT: subl %esi, %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: bitfield:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: addl %ecx, %ecx
+; X86-NEXT: andl $120, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ %s = lshr i32 %c, 2
+ %m = and i32 %s, 15
+ %shl = shl i32 %m, 3
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; A plain constant mask absorbs nothing - shl(and(c, 255), 2) stays two
+; operations either way - so here the fold does apply.
+define i32 @masked(i32 %x, i32 %c) {
+; X64-LABEL: masked:
+; X64: # %bb.0:
+; X64-NEXT: # kill: def $edi killed $edi def $rdi
+; X64-NEXT: movzbl %sil, %eax
+; X64-NEXT: negl %eax
+; X64-NEXT: leal (%rdi,%rax,4), %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: masked:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movzbl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: shll $2, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ %m = and i32 %c, 255
+ %shl = shl i32 %m, 2
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; shl(mul(y, 15), 2) is reassociated to mul(y, 60) first, so again no shl
+; survives to this combine and the multiply is left intact.
+define i32 @mul_const(i32 %x, i32 %y) {
+; X64-LABEL: mul_const:
+; X64: # %bb.0:
+; X64-NEXT: movl %edi, %eax
+; X64-NEXT: imull $60, %esi, %ecx
+; X64-NEXT: subl %ecx, %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: mul_const:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: imull $60, {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: retl
+ %m = mul i32 %y, 15
+ %shl = shl i32 %m, 2
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; shl(add(y, 15), 2) reassociates to add(shl(y, 2), 60) before this combine
+; runs, so by the time we see it Y is a bare value and the fold applies - the
+; constant lands in the LEA displacement.
+define i32 @add_const(i32 %x, i32 %y) {
+; X64-LABEL: add_const:
+; X64: # %bb.0:
+; X64-NEXT: # kill: def $esi killed $esi def $rsi
+; X64-NEXT: # kill: def $edi killed $edi def $rdi
+; X64-NEXT: negl %esi
+; X64-NEXT: leal -60(%rdi,%rsi,4), %eax
+; X64-NEXT: retq
+;
+; X86-LABEL: add_const:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: shll $2, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: addl $-60, %eax
+; X86-NEXT: retl
+ %a = add i32 %y, 15
+ %shl = shl i32 %a, 2
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; Same absorbable shift as @bitfield, but reaching us through a width change.
+; The reassociation sees through the trunc/zext, so this does not fold either.
+define i64 @bitfield_trunc(i64 %x, i64 %v) {
+; X64-LABEL: bitfield_trunc:
+; X64: # %bb.0:
+; X64-NEXT: movq %rdi, %rax
+; X64-NEXT: addl %esi, %esi
+; X64-NEXT: andl $120, %esi
+; X64-NEXT: subq %rsi, %rax
+; X64-NEXT: retq
+;
+; X86-LABEL: bitfield_trunc:
+; X86: # %bb.0:
+; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT: addl %ecx, %ecx
+; X86-NEXT: andl $120, %ecx
+; X86-NEXT: subl %ecx, %eax
+; X86-NEXT: sbbl $0, %edx
+; X86-NEXT: retl
+ %a = lshr i64 %v, 2
+ %b = trunc i64 %a to i32
+ %c = and i32 %b, 15
+ %d = zext i32 %c to i64
+ %shl = shl i64 %d, 3
+ %r = sub i64 %x, %shl
+ ret i64 %r
+}
diff --git a/llvm/test/CodeGen/X86/srem-vector-lkk.ll b/llvm/test/CodeGen/X86/srem-vector-lkk.ll
index be540704f6871..1a7e9475458bd 100644
--- a/llvm/test/CodeGen/X86/srem-vector-lkk.ll
+++ b/llvm/test/CodeGen/X86/srem-vector-lkk.ll
@@ -477,10 +477,10 @@ define <4 x i64> @fold_srem_i64(<4 x i64> %x) {
; SSE2-NEXT: sarq $4, %rdx
; SSE2-NEXT: addq %rax, %rdx
; SSE2-NEXT: leaq (%rdx,%rdx,2), %rax
-; SSE2-NEXT: shlq $3, %rax
-; SSE2-NEXT: subq %rax, %rdx
-; SSE2-NEXT: addq %rcx, %rdx
-; SSE2-NEXT: movq %rdx, %xmm1
+; SSE2-NEXT: negq %rax
+; SSE2-NEXT: leaq (%rdx,%rax,8), %rax
+; SSE2-NEXT: addq %rcx, %rax
+; SSE2-NEXT: movq %rax, %xmm1
; SSE2-NEXT: pshufd {{.*#+}} xmm2 = xmm2[2,3,2,3]
; SSE2-NEXT: movq %xmm2, %rcx
; SSE2-NEXT: movabsq $6966426675817289639, %rdx # imm = 0x60ADB826E5E517A7
@@ -522,10 +522,10 @@ define <4 x i64> @fold_srem_i64(<4 x i64> %x) {
; SSE4-NEXT: sarq $4, %rdx
; SSE4-NEXT: addq %rax, %rdx
; SSE4-NEXT: leaq (%rdx,%rdx,2), %rax
-; SSE4-NEXT: shlq $3, %rax
-; SSE4-NEXT: subq %rax, %rdx
-; SSE4-NEXT: addq %rcx, %rdx
-; SSE4-NEXT: movq %rdx, %xmm1
+; SSE4-NEXT: negq %rax
+; SSE4-NEXT: leaq (%rdx,%rax,8), %rax
+; SSE4-NEXT: addq %rcx, %rax
+; SSE4-NEXT: movq %rax, %xmm1
; SSE4-NEXT: pextrq $1, %xmm2, %rcx
; SSE4-NEXT: movabsq $6966426675817289639, %rdx # imm = 0x60ADB826E5E517A7
; SSE4-NEXT: movq %rcx, %rax
@@ -565,10 +565,10 @@ define <4 x i64> @fold_srem_i64(<4 x i64> %x) {
; AVX1-NEXT: sarq $4, %rdx
; AVX1-NEXT: addq %rax, %rdx
; AVX1-NEXT: leaq (%rdx,%rdx,2), %rax
-; AVX1-NEXT: shlq $3, %rax
-; AVX1-NEXT: subq %rax, %rdx
-; AVX1-NEXT: addq %rcx, %rdx
-; AVX1-NEXT: vmovq %rdx, %xmm2
+; AVX1-NEXT: negq %rax
+; AVX1-NEXT: leaq (%rdx,%rax,8), %rax
+; AVX1-NEXT: addq %rcx, %rax
+; AVX1-NEXT: vmovq %rax, %xmm2
; AVX1-NEXT: vpextrq $1, %xmm1, %rcx
; AVX1-NEXT: movabsq $6966426675817289639, %rdx # imm = 0x60ADB826E5E517A7
; AVX1-NEXT: movq %rcx, %rax
diff --git a/llvm/test/CodeGen/X86/urem-vector-lkk.ll b/llvm/test/CodeGen/X86/urem-vector-lkk.ll
index 7b3d0c9e12cfc..20fe4c0c12075 100644
--- a/llvm/test/CodeGen/X86/urem-vector-lkk.ll
+++ b/llvm/test/CodeGen/X86/urem-vector-lkk.ll
@@ -263,8 +263,8 @@ define <4 x i64> @fold_urem_i64(<4 x i64> %x) {
; SSE2-NEXT: addq %rdx, %rax
; SSE2-NEXT: shrq $4, %rax
; SSE2-NEXT: leaq (%rax,%rax,2), %rdx
-; SSE2-NEXT: shlq $3, %rdx
-; SSE2-NEXT: subq %rdx, %rax
+; SSE2-NEXT: negq %rdx
+; SSE2-NEXT: leaq (%rax,%rdx,8), %rax
; SSE2-NEXT: addq %rcx, %rax
; SSE2-NEXT: movq %rax, %xmm1
; SSE2-NEXT: pshufd {{.*#+}} xmm2 = xmm2[2,3,2,3]
@@ -302,8 +302,8 @@ define <4 x i64> @fold_urem_i64(<4 x i64> %x) {
; SSE4-NEXT: addq %rdx, %rax
; SSE4-NEXT: shrq $4, %rax
; SSE4-NEXT: leaq (%rax,%rax,2), %rdx
-; SSE4-NEXT: shlq $3, %rdx
-; SSE4-NEXT: subq %rdx, %rax
+; SSE4-NEXT: negq %rdx
+; SSE4-NEXT: leaq (%rax,%rdx,8), %rax
; SSE4-NEXT: addq %rcx, %rax
; SSE4-NEXT: movq %rax, %xmm2
; SSE4-NEXT: pextrq $1, %xmm1, %rcx
@@ -341,8 +341,8 @@ define <4 x i64> @fold_urem_i64(<4 x i64> %x) {
; AVX1-NEXT: addq %rdx, %rax
; AVX1-NEXT: shrq $4, %rax
; AVX1-NEXT: leaq (%rax,%rax,2), %rdx
-; AVX1-NEXT: shlq $3, %rdx
-; AVX1-NEXT: subq %rdx, %rax
+; AVX1-NEXT: negq %rdx
+; AVX1-NEXT: leaq (%rax,%rdx,8), %rax
; AVX1-NEXT: addq %rcx, %rax
; AVX1-NEXT: vmovq %rax, %xmm2
; AVX1-NEXT: vpextrq $1, %xmm1, %rcx
>From 8f7a715d8ffd31d976047e31bc2bab8b0a4d18f9 Mon Sep 17 00:00:00 2001
From: Nikita Taranov <nickita.taranov at gmail.com>
Date: Sun, 9 Aug 2026 19:43:48 +0000
Subject: [PATCH 2/4] better
---
llvm/include/llvm/CodeGen/TargetLowering.h | 5 -----
llvm/lib/Target/X86/X86ISelLowering.cpp | 15 ++-------------
2 files changed, 2 insertions(+), 18 deletions(-)
diff --git a/llvm/include/llvm/CodeGen/TargetLowering.h b/llvm/include/llvm/CodeGen/TargetLowering.h
index c4ac7ec484f9b..fdacfe61d638b 100644
--- a/llvm/include/llvm/CodeGen/TargetLowering.h
+++ b/llvm/include/llvm/CodeGen/TargetLowering.h
@@ -968,11 +968,6 @@ class LLVM_ABI TargetLoweringBase {
/// the canonical (sub x, (shl y, n)). Targets with a scaled-index addressing
/// mode (e.g. x86 LEA) can fold the whole add+shl into one instruction, which
/// pays for the extra negate. \p ShAmt is the shift amount operand.
- ///
- /// Only consulted once the DAGCombiner reaches AfterLegalizeDAG: keeping the
- /// negate in place stops (shl y, n) from being reassociated into y's own
- /// computation, so the canonicalization has to run in the earlier phases for
- /// that to get its chance first.
virtual bool preferNegShlAddToSubShl(EVT VT, SDValue ShAmt) const {
return false;
}
diff --git a/llvm/lib/Target/X86/X86ISelLowering.cpp b/llvm/lib/Target/X86/X86ISelLowering.cpp
index be8370c9bf413..2383ae39b9169 100644
--- a/llvm/lib/Target/X86/X86ISelLowering.cpp
+++ b/llvm/lib/Target/X86/X86ISelLowering.cpp
@@ -63465,19 +63465,8 @@ SDValue X86TargetLowering::PerformDAGCombine(SDNode *N,
// when x is still live because SUB is two-address - so this saves an
// instruction and two bytes exactly when that MOV is needed, and breaks even
// otherwise.
-//
-// The LEA normally has no displacement and so is not a "3-operand LEA" for
-// X86FixupLEAs, though reassociation can fold a constant into one (see
-// @add_const in neg-shl-lea.ll); no subtarget splits it either way.
-//
-// What the saved MOV is worth varies more than the LEA does. By LLVM's
-// scheduling models it is a uop on Haswell/Skylake-class Intel and on AMD up
-// to Zen2, plus Jaguar, Bulldozer and Silvermont. It is nothing on Zen3 and
-// later, which charge an LEA with scale != 1 two uops, and nothing on Sapphire
-// Rapids, which eliminates reg-reg MOVs at rename. With APX the SUB is not
-// two-address to begin with and no MOV is needed at all. So on the newest
-// cores this is neutral rather than negative, and the size win holds
-// everywhere.
+// What the saved MOV is worth varies more than the LEA does. On the newest
+// cores this is neutral rather than negative, and the size win holds everywhere.
bool X86TargetLowering::preferNegShlAddToSubShl(EVT VT, SDValue ShAmt) const {
// 64-bit mode only. The rewrite is a small win on instruction count but it
// perturbs register allocation, and with only six allocatable registers in
>From 09324e2283949bb8dc5a0467f78d2ccc0af8855b Mon Sep 17 00:00:00 2001
From: Nikita Taranov <nickita.taranov at gmail.com>
Date: Sun, 9 Aug 2026 20:01:17 +0000
Subject: [PATCH 3/4] [X86] Trim the neg+lea tests to what the fold actually
covers
The fold is 64-bit only, so the i686 RUN line in neg-shl-lea.ll produced 168
CHECK lines asserting that nothing happens - 38% of the file - and forced the
guard tests to use i32 so the 32-bit output stayed readable. Move that coverage
to neg-shl-lea-i686.ll, which keeps three functions: the plain shl form, the
multiply spelling from the issue, and the X-live-across-the-sub shape, whose
32-bit output still contains the MOV that the fold removes in 64-bit mode. That
demonstrates the gate rather than just asserting it.
Drop @shl0. A shift by zero is folded away long before combineSub runs, so
Op1.getOpcode() == ISD::SHL already fails and the shift-amount range check is
never reached; the function would pass with the lower bound deleted, so it was
testing nothing. No IR can reach that bound. @shl4 covers the upper bound for
real - shll $4 is right there in its output.
Also cover the multiply spelling at every usable scale. mul_form only tested
x + -4*y; add x + -2*y and x + -8*y, which reach the fold through the
mul -> shl path (there is no InstCombine in llc, so the mul arrives intact),
and x + -3*y as a negative - a non-power-of-two multiply never becomes a shl,
so this combine never sees it, and LLVM's mov+lea+sub already ties GCC's
lea+sub+lea at three instructions.
neg-shl-lea.ll: 443 -> 289 lines. No codegen change.
---
llvm/test/CodeGen/X86/neg-shl-lea-i686.ll | 50 +++++
llvm/test/CodeGen/X86/neg-shl-lea.ll | 250 +++++-----------------
2 files changed, 98 insertions(+), 202 deletions(-)
create mode 100644 llvm/test/CodeGen/X86/neg-shl-lea-i686.ll
diff --git a/llvm/test/CodeGen/X86/neg-shl-lea-i686.ll b/llvm/test/CodeGen/X86/neg-shl-lea-i686.ll
new file mode 100644
index 0000000000000..a7be37ad9edfb
--- /dev/null
+++ b/llvm/test/CodeGen/X86/neg-shl-lea-i686.ll
@@ -0,0 +1,50 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 6
+; RUN: llc < %s -mtriple=i686-unknown-linux-gnu | FileCheck %s
+
+; The neg+lea fold for sub(X, shl(Y, C)) is 64-bit mode only - see
+; neg-shl-lea.ll for the positive tests. With six allocatable registers the
+; extra live value regularly costs a spill, so 32-bit mode keeps SHL + SUB.
+
+define i32 @shl2(i32 %x, i32 %y) {
+; CHECK-LABEL: shl2:
+; CHECK: # %bb.0:
+; CHECK-NEXT: movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT: shll $2, %ecx
+; CHECK-NEXT: subl %ecx, %eax
+; CHECK-NEXT: retl
+ %shl = shl i32 %y, 2
+ %r = sub i32 %x, %shl
+ ret i32 %r
+}
+
+; The spelling from the issue: x + -4*y.
+define i32 @mul_form(i32 %x, i32 %y) {
+; CHECK-LABEL: mul_form:
+; CHECK: # %bb.0:
+; CHECK-NEXT: movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT: shll $2, %ecx
+; CHECK-NEXT: subl %ecx, %eax
+; CHECK-NEXT: retl
+ %m = mul i32 %y, -4
+ %r = add i32 %x, %m
+ ret i32 %r
+}
+
+; X live across the sub - the shape the fold targets in 64-bit mode.
+define i32 @x_live_after(i32 %x, i32 %y) {
+; CHECK-LABEL: x_live_after:
+; CHECK: # %bb.0:
+; CHECK-NEXT: movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT: movl {{[0-9]+}}(%esp), %edx
+; CHECK-NEXT: shll $3, %edx
+; CHECK-NEXT: movl %ecx, %eax
+; CHECK-NEXT: subl %edx, %eax
+; CHECK-NEXT: xorl %ecx, %eax
+; CHECK-NEXT: retl
+ %shl = shl i32 %y, 3
+ %s = sub i32 %x, %shl
+ %r = xor i32 %s, %x
+ ret i32 %r
+}
diff --git a/llvm/test/CodeGen/X86/neg-shl-lea.ll b/llvm/test/CodeGen/X86/neg-shl-lea.ll
index 3f5878595bb12..f65f41123aa47 100644
--- a/llvm/test/CodeGen/X86/neg-shl-lea.ll
+++ b/llvm/test/CodeGen/X86/neg-shl-lea.ll
@@ -1,13 +1,10 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 6
; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --check-prefix=X64
-; RUN: llc < %s -mtriple=i686-unknown-linux-gnu | FileCheck %s --check-prefix=X86
; sub(X, shl(Y, C)) with C in [1,3] becomes neg + a scaled-index LEA, saving
; the MOV that the destructive SUB would otherwise need. See issue #37287.
;
-; The fold is 64-bit mode only, so every X86 check below is a negative one.
-; The guard tests use i32 to keep the 32-bit output readable; i64 in 32-bit
-; mode is split into register pairs and says nothing extra.
+; The fold is 64-bit mode only; neg-shl-lea-i686.ll covers that gate.
define i64 @shl1(i64 %x, i64 %y) {
; X64-LABEL: shl1:
@@ -15,23 +12,6 @@ define i64 @shl1(i64 %x, i64 %y) {
; X64-NEXT: negq %rsi
; X64-NEXT: leaq (%rdi,%rsi,2), %rax
; X64-NEXT: retq
-;
-; X86-LABEL: shl1:
-; X86: # %bb.0:
-; X86-NEXT: pushl %esi
-; X86-NEXT: .cfi_def_cfa_offset 8
-; X86-NEXT: .cfi_offset %esi, -8
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT: shldl $1, %ecx, %esi
-; X86-NEXT: addl %ecx, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: sbbl %esi, %edx
-; X86-NEXT: popl %esi
-; X86-NEXT: .cfi_def_cfa_offset 4
-; X86-NEXT: retl
%shl = shl i64 %y, 1
%r = sub i64 %x, %shl
ret i64 %r
@@ -43,23 +23,6 @@ define i64 @shl2(i64 %x, i64 %y) {
; X64-NEXT: negq %rsi
; X64-NEXT: leaq (%rdi,%rsi,4), %rax
; X64-NEXT: retq
-;
-; X86-LABEL: shl2:
-; X86: # %bb.0:
-; X86-NEXT: pushl %esi
-; X86-NEXT: .cfi_def_cfa_offset 8
-; X86-NEXT: .cfi_offset %esi, -8
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT: shldl $2, %ecx, %esi
-; X86-NEXT: shll $2, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: sbbl %esi, %edx
-; X86-NEXT: popl %esi
-; X86-NEXT: .cfi_def_cfa_offset 4
-; X86-NEXT: retl
%shl = shl i64 %y, 2
%r = sub i64 %x, %shl
ret i64 %r
@@ -71,23 +34,6 @@ define i64 @shl3(i64 %x, i64 %y) {
; X64-NEXT: negq %rsi
; X64-NEXT: leaq (%rdi,%rsi,8), %rax
; X64-NEXT: retq
-;
-; X86-LABEL: shl3:
-; X86: # %bb.0:
-; X86-NEXT: pushl %esi
-; X86-NEXT: .cfi_def_cfa_offset 8
-; X86-NEXT: .cfi_offset %esi, -8
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT: shldl $3, %ecx, %esi
-; X86-NEXT: shll $3, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: sbbl %esi, %edx
-; X86-NEXT: popl %esi
-; X86-NEXT: .cfi_def_cfa_offset 4
-; X86-NEXT: retl
%shl = shl i64 %y, 3
%r = sub i64 %x, %shl
ret i64 %r
@@ -101,20 +47,14 @@ define i32 @shl2_i32(i32 %x, i32 %y) {
; X64-NEXT: negl %esi
; X64-NEXT: leal (%rdi,%rsi,4), %eax
; X64-NEXT: retq
-;
-; X86-LABEL: shl2_i32:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: shll $2, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
%shl = shl i32 %y, 2
%r = sub i32 %x, %shl
ret i32 %r
}
-; The multiply spelling from the original report: x + -4*y.
+; The multiply spelling from the original report, at each usable scale. The mul
+; reaches the DAG intact here (there is no InstCombine in llc), so these cover
+; the mul -> shl -> fold path rather than repeating the shl tests above.
define i32 @mul_form(i32 %x, i32 %y) {
; X64-LABEL: mul_form:
; X64: # %bb.0:
@@ -123,19 +63,53 @@ define i32 @mul_form(i32 %x, i32 %y) {
; X64-NEXT: negl %esi
; X64-NEXT: leal (%rdi,%rsi,4), %eax
; X64-NEXT: retq
-;
-; X86-LABEL: mul_form:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: shll $2, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
%m = mul i32 %y, -4
%r = add i32 %x, %m
ret i32 %r
}
+define i32 @mul_form_2(i32 %x, i32 %y) {
+; X64-LABEL: mul_form_2:
+; X64: # %bb.0:
+; X64-NEXT: # kill: def $esi killed $esi def $rsi
+; X64-NEXT: # kill: def $edi killed $edi def $rdi
+; X64-NEXT: negl %esi
+; X64-NEXT: leal (%rdi,%rsi,2), %eax
+; X64-NEXT: retq
+ %m = mul i32 %y, -2
+ %r = add i32 %x, %m
+ ret i32 %r
+}
+
+define i32 @mul_form_8(i32 %x, i32 %y) {
+; X64-LABEL: mul_form_8:
+; X64: # %bb.0:
+; X64-NEXT: # kill: def $esi killed $esi def $rsi
+; X64-NEXT: # kill: def $edi killed $edi def $rdi
+; X64-NEXT: negl %esi
+; X64-NEXT: leal (%rdi,%rsi,8), %eax
+; X64-NEXT: retq
+ %m = mul i32 %y, -8
+ %r = add i32 %x, %m
+ ret i32 %r
+}
+
+; -3 is not a power of two, so the mul never becomes a shl and this combine
+; never sees it. Left alone deliberately: LLVM's mov+lea+sub matches GCC's
+; lea+sub+lea at three instructions.
+define i32 @mul_form_3(i32 %x, i32 %y) {
+; X64-LABEL: mul_form_3:
+; X64: # %bb.0:
+; X64-NEXT: # kill: def $esi killed $esi def $rsi
+; X64-NEXT: movl %edi, %eax
+; X64-NEXT: leal (%rsi,%rsi,2), %ecx
+; X64-NEXT: subl %ecx, %eax
+; X64-NEXT: retq
+ %m = mul i32 %y, -3
+ %r = add i32 %x, %m
+ ret i32 %r
+}
+
; The other canonical spelling, add(x, shl(0 - y, n)). This one reaches the
; fold through the TargetLowering hook rather than through combineSub, so it
; covers the second call site.
@@ -147,14 +121,6 @@ define i32 @add_neg_form(i32 %x, i32 %y) {
; X64-NEXT: negl %esi
; X64-NEXT: leal (%rdi,%rsi,8), %eax
; X64-NEXT: retq
-;
-; X86-LABEL: add_neg_form:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: shll $3, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
%n = sub i32 0, %y
%shl = shl i32 %n, 3
%r = add i32 %x, %shl
@@ -170,32 +136,6 @@ define i64 @x_live_after(i64 %x, i64 %y) {
; X64-NEXT: leaq (%rdi,%rsi,8), %rax
; X64-NEXT: xorq %rdi, %rax
; X64-NEXT: retq
-;
-; X86-LABEL: x_live_after:
-; X86: # %bb.0:
-; X86-NEXT: pushl %edi
-; X86-NEXT: .cfi_def_cfa_offset 8
-; X86-NEXT: pushl %esi
-; X86-NEXT: .cfi_def_cfa_offset 12
-; X86-NEXT: .cfi_offset %esi, -12
-; X86-NEXT: .cfi_offset %edi, -8
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %edi
-; X86-NEXT: shldl $3, %edx, %edi
-; X86-NEXT: shll $3, %edx
-; X86-NEXT: movl %ecx, %eax
-; X86-NEXT: subl %edx, %eax
-; X86-NEXT: movl %esi, %edx
-; X86-NEXT: sbbl %edi, %edx
-; X86-NEXT: xorl %ecx, %eax
-; X86-NEXT: xorl %esi, %edx
-; X86-NEXT: popl %esi
-; X86-NEXT: .cfi_def_cfa_offset 8
-; X86-NEXT: popl %edi
-; X86-NEXT: .cfi_def_cfa_offset 4
-; X86-NEXT: retl
%shl = shl i64 %y, 3
%s = sub i64 %x, %shl
%r = xor i64 %s, %x
@@ -204,24 +144,8 @@ define i64 @x_live_after(i64 %x, i64 %y) {
;; Negative tests.
-; C = 0 and C >= 4 are outside the LEA scale range.
-define i32 @shl0(i32 %x, i32 %y) {
-; X64-LABEL: shl0:
-; X64: # %bb.0:
-; X64-NEXT: movl %edi, %eax
-; X64-NEXT: subl %esi, %eax
-; X64-NEXT: retq
-;
-; X86-LABEL: shl0:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: subl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: retl
- %shl = shl i32 %y, 0
- %r = sub i32 %x, %shl
- ret i32 %r
-}
-
+; C >= 4 is outside the LEA scale range. (C = 0 cannot reach the combine -
+; shl by zero is folded away first - so there is nothing to test there.)
define i32 @shl4(i32 %x, i32 %y) {
; X64-LABEL: shl4:
; X64: # %bb.0:
@@ -229,14 +153,6 @@ define i32 @shl4(i32 %x, i32 %y) {
; X64-NEXT: shll $4, %esi
; X64-NEXT: subl %esi, %eax
; X64-NEXT: retq
-;
-; X86-LABEL: shl4:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: shll $4, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
%shl = shl i32 %y, 4
%r = sub i32 %x, %shl
ret i32 %r
@@ -252,16 +168,6 @@ define i32 @y_multi_use(i32 %x, i32 %y, ptr %p) {
; X64-NEXT: shll $2, %esi
; X64-NEXT: subl %esi, %eax
; X64-NEXT: retq
-;
-; X86-LABEL: y_multi_use:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT: movl %ecx, (%edx)
-; X86-NEXT: shll $2, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
store i32 %y, ptr %p
%shl = shl i32 %y, 2
%r = sub i32 %x, %shl
@@ -277,13 +183,6 @@ define i32 @x_minus_x_shl(i32 %x) {
; X64-NEXT: subl %ecx, %eax
; X64-NEXT: # kill: def $eax killed $eax killed $rax
; X64-NEXT: retq
-;
-; X86-LABEL: x_minus_x_shl:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: leal (,%eax,8), %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
%shl = shl i32 %x, 3
%r = sub i32 %x, %shl
ret i32 %r
@@ -298,15 +197,6 @@ define i32 @x_is_load(ptr %p, i32 %y) {
; X64-NEXT: shll $2, %esi
; X64-NEXT: subl %esi, %eax
; X64-NEXT: retq
-;
-; X86-LABEL: x_is_load:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl (%eax), %eax
-; X86-NEXT: shll $2, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
%x = load i32, ptr %p
%shl = shl i32 %y, 2
%r = sub i32 %x, %shl
@@ -324,15 +214,6 @@ define i32 @bitfield(i32 %x, i32 %c) {
; X64-NEXT: andl $120, %esi
; X64-NEXT: subl %esi, %eax
; X64-NEXT: retq
-;
-; X86-LABEL: bitfield:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: addl %ecx, %ecx
-; X86-NEXT: andl $120, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
%s = lshr i32 %c, 2
%m = and i32 %s, 15
%shl = shl i32 %m, 3
@@ -350,14 +231,6 @@ define i32 @masked(i32 %x, i32 %c) {
; X64-NEXT: negl %eax
; X64-NEXT: leal (%rdi,%rax,4), %eax
; X64-NEXT: retq
-;
-; X86-LABEL: masked:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movzbl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: shll $2, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
%m = and i32 %c, 255
%shl = shl i32 %m, 2
%r = sub i32 %x, %shl
@@ -373,13 +246,6 @@ define i32 @mul_const(i32 %x, i32 %y) {
; X64-NEXT: imull $60, %esi, %ecx
; X64-NEXT: subl %ecx, %eax
; X64-NEXT: retq
-;
-; X86-LABEL: mul_const:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: imull $60, {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: retl
%m = mul i32 %y, 15
%shl = shl i32 %m, 2
%r = sub i32 %x, %shl
@@ -397,15 +263,6 @@ define i32 @add_const(i32 %x, i32 %y) {
; X64-NEXT: negl %esi
; X64-NEXT: leal -60(%rdi,%rsi,4), %eax
; X64-NEXT: retq
-;
-; X86-LABEL: add_const:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: shll $2, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: addl $-60, %eax
-; X86-NEXT: retl
%a = add i32 %y, 15
%shl = shl i32 %a, 2
%r = sub i32 %x, %shl
@@ -422,17 +279,6 @@ define i64 @bitfield_trunc(i64 %x, i64 %v) {
; X64-NEXT: andl $120, %esi
; X64-NEXT: subq %rsi, %rax
; X64-NEXT: retq
-;
-; X86-LABEL: bitfield_trunc:
-; X86: # %bb.0:
-; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT: movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT: addl %ecx, %ecx
-; X86-NEXT: andl $120, %ecx
-; X86-NEXT: subl %ecx, %eax
-; X86-NEXT: sbbl $0, %edx
-; X86-NEXT: retl
%a = lshr i64 %v, 2
%b = trunc i64 %a to i32
%c = and i32 %b, 15
>From eedf0f45ae6030ed1f3625129f546dba0a60e5b7 Mon Sep 17 00:00:00 2001
From: Nikita Taranov <nikita.taranov at clickhouse.com>
Date: Mon, 10 Aug 2026 09:57:49 +0000
Subject: [PATCH 4/4] fix style
---
llvm/lib/Target/X86/X86ISelLowering.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/llvm/lib/Target/X86/X86ISelLowering.cpp b/llvm/lib/Target/X86/X86ISelLowering.cpp
index 2383ae39b9169..9f0f7b7926ee6 100644
--- a/llvm/lib/Target/X86/X86ISelLowering.cpp
+++ b/llvm/lib/Target/X86/X86ISelLowering.cpp
@@ -63466,7 +63466,7 @@ SDValue X86TargetLowering::PerformDAGCombine(SDNode *N,
// instruction and two bytes exactly when that MOV is needed, and breaks even
// otherwise.
// What the saved MOV is worth varies more than the LEA does. On the newest
-// cores this is neutral rather than negative, and the size win holds everywhere.
+// cores this is neutral rather than negative, and the size win holds.
bool X86TargetLowering::preferNegShlAddToSubShl(EVT VT, SDValue ShAmt) const {
// 64-bit mode only. The rewrite is a small win on instruction count but it
// perturbs register allocation, and with only six allocatable registers in
More information about the llvm-commits
mailing list