[llvm] [X86][CostModel] Free a clean narrow zext used as a GEP index (PR #216256)
via llvm-commits
llvm-commits at lists.llvm.org
Thu Aug 13 23:28:46 PDT 2026
https://github.com/mbhade-amd created https://github.com/llvm/llvm-project/pull/216256
`getCastInstrCost` already treats a narrow (i8/i16) zext of a load as free (the extension folds into the load). This extends the same movzx-free reasoning to any narrow zext whose single use is a **GEP index** with a provably-clean source — a load, a `zeroext` argument, or known-zero high bits. On x86-64 `[base + index*scale + disp]` reads the index at full width, so a clean narrow index needs no `movzx`. A dirty source (e.g. an `i16 add` used only as an index) still needs one and is unchanged. The cost is consumed by SimplifyCFG's two-entry-PHI fold budget (`TCK_SizeAndLatency`); overcharging a clean index by one kept small select
diamonds branched on x86.
Test: `llvm/test/Analysis/CostModel/X86/zext-gep-index.ll` — fails on trunk
(clean known-bits index reports 1), passes with the patch.
>From e55236ab62973f0c99fb34209bc166d42039a466 Mon Sep 17 00:00:00 2001
From: mbhade <mbhade at amd.com>
Date: Fri, 14 Aug 2026 11:45:52 +0530
Subject: [PATCH] [X86][CostModel] Free a clean narrow zext used as a GEP index
getCastInstrCost already prices a narrow (i8/i16) zext of a load at 0.
Extend that to any narrow zext whose single use is a GEP index and whose
source is provably clean (a load, a zeroext argument, or a value with
known-zero high bits): x86's SIB [base + index*scale] reads the index at
full width, so such an index needs no movzx. A dirty source (e.g. an i16
add used only as an index) still needs one and keeps its normal cost.
The extend cost feeds SimplifyCFG's two-entry-PHI fold budget
(TCK_SizeAndLatency); overpricing it kept clean select diamonds branched.
---
.../lib/Target/X86/X86TargetTransformInfo.cpp | 33 ++++
.../Analysis/CostModel/X86/zext-gep-index.ll | 141 ++++++++++++++++++
2 files changed, 174 insertions(+)
create mode 100644 llvm/test/Analysis/CostModel/X86/zext-gep-index.ll
diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index 828d79ab0cb45..1cdd6cbd2ff41 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -51,12 +51,14 @@
#include "X86TargetTransformInfo.h"
#include "llvm/ADT/SmallBitVector.h"
#include "llvm/Analysis/TargetTransformInfo.h"
+#include "llvm/Analysis/ValueTracking.h"
#include "llvm/CodeGen/Analysis.h"
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/CodeGen/CostTable.h"
#include "llvm/CodeGen/TargetLowering.h"
#include "llvm/IR/InstIterator.h"
#include "llvm/IR/IntrinsicInst.h"
+#include "llvm/Support/KnownBits.h"
#include <optional>
using namespace llvm;
@@ -2429,6 +2431,37 @@ InstructionCost X86TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst,
int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode");
+ // A narrow (i8/i16) zero-extension used as a GEP *index* can be folded into
+ // the addressing mode of the consuming memory op, but only if the source is
+ // already materialised zero-extended in a full register. X86's SIB form
+ // [base + index*scale + disp] reads the index at full width and does NOT
+ // zero-extend a narrow index (unlike AArch64's uxtw-extended addressing), so
+ // a "dirty" narrow source (e.g. an i16 add result used only as an index)
+ // still needs a dedicated movzx and is not free. Price it as free only with
+ // positive evidence that no movzx is required.
+ if (ISD == ISD::ZERO_EXTEND && I && I->hasOneUse() && Src->isIntegerTy() &&
+ Src->getScalarSizeInBits() < 32) {
+ const Use &U = *I->use_begin();
+ if (isa<GetElementPtrInst>(U.getUser()) &&
+ U.getOperandNo() != GetElementPtrInst::getPointerOperandIndex()) {
+ const Value *Op = I->getOperand(0);
+ // Clean sources: an extending load, a zeroext argument, or a value whose
+ // high bits are provably zero (e.g. from a shift/mask). These mirror the
+ // proof-based reasoning the middle end uses elsewhere (ValueTracking and
+ // InstCombine's canEvaluateZExtd); we intentionally do NOT treat a merely
+ // multiply-used operand as clean, since that is a guess rather than proof.
+ bool CleanSource = isa<LoadInst>(Op);
+ if (!CleanSource)
+ if (const auto *A = dyn_cast<Argument>(Op))
+ CleanSource = A->hasAttribute(Attribute::ZExt);
+ if (!CleanSource)
+ CleanSource = computeKnownBits(Op, I->getDataLayout(), /*AC=*/nullptr, I)
+ .countMinLeadingZeros() > 0;
+ if (CleanSource)
+ return 0;
+ }
+ }
+
// The cost tables include both specific, custom (non-legal) src/dst type
// conversions and generic, legalized types. We test for customs first, before
// falling back to legalization.
diff --git a/llvm/test/Analysis/CostModel/X86/zext-gep-index.ll b/llvm/test/Analysis/CostModel/X86/zext-gep-index.ll
new file mode 100644
index 0000000000000..b8574873d74dc
--- /dev/null
+++ b/llvm/test/Analysis/CostModel/X86/zext-gep-index.ll
@@ -0,0 +1,141 @@
+; NOTE: Assertions have been autogenerated by utils/update_analyze_test_checks.py
+; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -cost-kind=size-latency -mtriple=x86_64-unknown-linux-gnu | FileCheck %s
+
+; A narrow (i8/i16) zero-extend whose single use is a GEP *index* operand folds
+; into the addressing mode of the consuming memory op (no movzx), but only when
+; the source is already materialised zero-extended in a full register. X86's SIB
+; form [base + index*scale + disp] reads the index at full width and does NOT
+; zero-extend a narrow index (unlike AArch64's uxtw-extended addressing), so a
+; "dirty" narrow source still needs a dedicated movzx and is not free. The cost
+; model prices the zext at 0 only with positive proof that no movzx is required.
+;
+; This cost feeds SimplifyCFG's two-entry-PHI fold budget, which queries
+; TCK_SizeAndLatency -- hence -cost-kind=size-latency here.
+
+ at tab = external global [0 x i8]
+
+;; --- Clean sources: no movzx is emitted, so the zext is free (cost 0). ---
+
+; Extending load: the load already leaves a zero-extended value in a full
+; register, so the index folds into the table load with no extra instruction.
+define i8 @clean_i16_load(ptr %q) {
+; CHECK-LABEL: 'clean_i16_load'
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %x = load i16, ptr %q, align 2
+; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %z = zext i16 %x to i64
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %p = getelementptr inbounds i8, ptr @tab, i64 %z
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %v = load i8, ptr %p, align 1
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i8 %v
+;
+ %x = load i16, ptr %q, align 2
+ %z = zext i16 %x to i64
+ %p = getelementptr inbounds i8, ptr @tab, i64 %z
+ %v = load i8, ptr %p, align 1
+ ret i8 %v
+}
+
+; Same for an i8 extending load.
+define i8 @clean_i8_load(ptr %q) {
+; CHECK-LABEL: 'clean_i8_load'
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %x = load i8, ptr %q, align 1
+; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %z = zext i8 %x to i64
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %p = getelementptr inbounds i8, ptr @tab, i64 %z
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %v = load i8, ptr %p, align 1
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i8 %v
+;
+ %x = load i8, ptr %q, align 1
+ %z = zext i8 %x to i64
+ %p = getelementptr inbounds i8, ptr @tab, i64 %z
+ %v = load i8, ptr %p, align 1
+ ret i8 %v
+}
+
+; High bits provably zero from a shift: no movzx needed.
+define i8 @clean_known_bits(i16 zeroext %x) {
+; CHECK-LABEL: 'clean_known_bits'
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %sh = lshr i16 %x, 7
+; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %z = zext i16 %sh to i64
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %p = getelementptr inbounds i8, ptr @tab, i64 %z
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %v = load i8, ptr %p, align 1
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i8 %v
+;
+ %sh = lshr i16 %x, 7
+ %z = zext i16 %sh to i64
+ %p = getelementptr inbounds i8, ptr @tab, i64 %z
+ %v = load i8, ptr %p, align 1
+ ret i8 %v
+}
+
+;; --- Dirty / non-foldable sources: the discount must NOT apply (cost 1). ---
+
+; Dirty i16 add used only as an index: bits 16+ of the register are non-zero,
+; so a dedicated movzwl is emitted.
+define i8 @dirty_i16_add(i16 zeroext %a, i16 zeroext %b) {
+; CHECK-LABEL: 'dirty_i16_add'
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %d = add i16 %a, %b
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %z = zext i16 %d to i64
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %p = getelementptr inbounds i8, ptr @tab, i64 %z
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %v = load i8, ptr %p, align 1
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i8 %v
+;
+ %d = add i16 %a, %b
+ %z = zext i16 %d to i64
+ %p = getelementptr inbounds i8, ptr @tab, i64 %z
+ %v = load i8, ptr %p, align 1
+ ret i8 %v
+}
+
+; The dirty add is also used by an lshr. We do NOT speculate that a shared use
+; keeps the value clean: the index zext of the dirty add still costs a movzwl,
+; while the lshr arm has provably-zero high bits and stays free.
+define i8 @dirty_add_shared(ptr %t, i16 zeroext %a, i16 zeroext %b) {
+; CHECK-LABEL: 'dirty_add_shared'
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %d = add i16 %a, %b
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %z = zext i16 %d to i64
+; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %p = getelementptr inbounds i8, ptr %t, i64 %z
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %v = load i8, ptr %p, align 1
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %sh = lshr i16 %d, 7
+; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %ze = zext i16 %sh to i64
+; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %pe = getelementptr inbounds i8, ptr %t, i64 %ze
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %ve = load i8, ptr %pe, align 1
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %sum = add i8 %v, %ve
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i8 %sum
+;
+ %d = add i16 %a, %b
+ %z = zext i16 %d to i64
+ %p = getelementptr inbounds i8, ptr %t, i64 %z
+ %v = load i8, ptr %p, align 1
+ %sh = lshr i16 %d, 7
+ %ze = zext i16 %sh to i64
+ %pe = getelementptr inbounds i8, ptr %t, i64 %ze
+ %ve = load i8, ptr %pe, align 1
+ %sum = add i8 %v, %ve
+ ret i8 %sum
+}
+
+; zext not consumed by a GEP (feeds a return): normal cost.
+define i64 @zext_i16_ret(i16 zeroext %x) {
+; CHECK-LABEL: 'zext_i16_ret'
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %z = zext i16 %x to i64
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i64 %z
+;
+ %z = zext i16 %x to i64
+ ret i64 %z
+}
+
+; zext has more than one use: not a pure foldable index, normal cost.
+define i64 @zext_i16_multiuse(i16 zeroext %x) {
+; CHECK-LABEL: 'zext_i16_multiuse'
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %z = zext i16 %x to i64
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %p = getelementptr inbounds i8, ptr @tab, i64 %z
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %v = load i8, ptr %p, align 1
+; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %e = zext i8 %v to i64
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %s = add i64 %z, %e
+; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i64 %s
+;
+ %z = zext i16 %x to i64
+ %p = getelementptr inbounds i8, ptr @tab, i64 %z
+ %v = load i8, ptr %p, align 1
+ %e = zext i8 %v to i64
+ %s = add i64 %z, %e
+ ret i64 %s
+}
More information about the llvm-commits
mailing list