[llvm] [X86][CostModel] Add vXi64 divide/remainder-by-constant costs (PR #208491)
via llvm-commits
llvm-commits at lists.llvm.org
Thu Jul 9 08:48:40 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-backend-x86
Author: Rito Takeuchi (Licht-T)
<details>
<summary>Changes</summary>
### What this does
Follow-up to #<!-- -->206983, which lowered `vXi64` `MULHU`/`MULHS`. Modified the cost model to enable the vectorization:
1. `SDIV`/`SREM` for `v4i64`/`v8i64`, gated on AVX512DQ
`MULHS`'s low multiply is `vpmullq`. Without DQ, it scalarizes and can be a performance regression. So, this PR adds `AVX512DQ{UniformConst,Const}` tables which are consulted before the AVX512/AVX2 ones. Then, non-DQ targets fall through to the scalarized cost.
2. `UDIV`/`UREM` for `v4i64`/`v8i64` in the AVX2 and AVX512
For `UREM`, the multiply-back in the remainder `x - (x/C)*C` is a single `vpmullq` with DQ, but another `vpmuludq` schoolbook without it. So, the cheap DQ `UREM` entries in `AVX512DQ{UniformConst,Const}` overrides more expensive AVX512/AVX2 ones.
### Performance
>From the bare-metal matrix in https://github.com/llvm/llvm-project/pull/206983#issuecomment-4871421534 (EC2 metal, per-target `-march`/`-mtune`, turbo off, pinned core; Skylake-X/Cascade Lake, Ice Lake-SP, Sapphire Rapids, Zen 4), vectorized vs scalar, which is exactly the gate implemented here (unsigned everywhere from AVX2, signed only with DQ):
| kernel | AVX2 | AVX-512 (DQ) |
|---|---|---|
| `uint64 / const` (`MULHU`) | 1.66 - 2.15x | 1.71 - 1.88x |
| `int64 / const` (`MULHS`) | 0.75 - 0.82x (Intel) | 1.18 - 2.11x |
### How the numbers were calibrated
Values are derived from the generated sequences: instruction counts of the real codegen, minus loop-invariant constant materialization, and cross-checked against `llvm-mca` reciprocal throughput. The instruction count varies with the divisor. So, the counts below are the range over a small (`7`) and a full-width (`1000000007`) divisor, and the mca figures are for the former.
Compiler Explorer: https://godbolt.org/z/c3v6YnE1c
| entry | loop-resident instrs | mca RThru | this patch |
|---|---|---|---|
| `udiv v4i64` uniform | 16 - 19 | 5.0 | 15 |
| `udiv v4i64` non-uniform | 16 - 23 | 6.5 | 22 |
| `sdiv v4i64` uniform (DQ) | 18 - 20 | 5.0 | 15 |
| `sdiv v4i64` non-uniform (DQ) | 18 - 21 | 6.0 | 19 |
| (existing) `udiv v8i32` uniform | 10 | 2.0 | 5 |
| (existing) `udiv v8i32` non-uniform | 11 | 2.5 | 15 |
### Testing
* `Analysis/CostModel/X86/{div,rem}.ll` regenerated, plus a new `+avx512f,+avx512dq` RUN line so the DQ-gated costs are actually asserted.
* New `Transforms/LoopVectorize/X86/vector-idiv-by-const.ll` pins the decision matrix above; for the signed loops three of the five configurations assert the loop must not vectorize.
* `Analysis/CostModel/X86`, `Transforms/LoopVectorize/X86` and `Transforms/SLPVectorizer/X86` are green, with no churn in existing tests. Each commit is green on its own.
Part of #<!-- -->37771.
### AI Usage Disclosure
This PR was prepared with the assistance of Claude Code.
---
Patch is 120.28 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/208491.diff
4 Files Affected:
- (modified) llvm/lib/Target/X86/X86TargetTransformInfo.cpp (+47)
- (modified) llvm/test/Analysis/CostModel/X86/div.ll (+253-24)
- (modified) llvm/test/Analysis/CostModel/X86/rem.ll (+215-24)
- (added) llvm/test/Transforms/LoopVectorize/X86/vector-idiv-by-const.ll (+125)
``````````diff
diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index 2993173d35edc..6493cd5c40c1f 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -422,6 +422,24 @@ InstructionCost X86TTIImpl::getArithmeticInstrCost(
if (auto KindCost = Entry->Cost[CostKind])
return LT.first * *KindCost;
+ static const CostKindTblEntry AVX512DQUniformConstCostTable[] = {
+ { ISD::SDIV, MVT::v4i64, { 15 } }, // vpmullq-based MULHS sequence
+ { ISD::SREM, MVT::v4i64, { 17 } }, // vpmullq-based MULHS+mul+sub sequence
+ { ISD::SDIV, MVT::v8i64, { 15 } }, // vpmullq-based MULHS sequence
+ { ISD::SREM, MVT::v8i64, { 17 } }, // vpmullq-based MULHS+mul+sub sequence
+ // The remainder's multiply-back is a single vpmullq with DQ, just like the
+ // pmulld the vXi32 entries above rely on. Without DQ it is another
+ // vpmuludq schoolbook, so the AVX512/AVX2 tables charge more.
+ { ISD::UREM, MVT::v4i64, { 17 } }, // MULHU + vpmullq + sub sequence
+ { ISD::UREM, MVT::v8i64, { 17 } }, // MULHU + vpmullq + sub sequence
+ };
+
+ if (Op2Info.isUniform() && Op2Info.isConstant() && ST->hasDQI())
+ if (const auto *Entry =
+ CostTableLookup(AVX512DQUniformConstCostTable, ISD, LT.second))
+ if (auto KindCost = Entry->Cost[CostKind])
+ return LT.first * *KindCost;
+
static const CostKindTblEntry AVX512UniformConstCostTable[] = {
{ ISD::SHL, MVT::v64i8, { 2, 12, 5, 6 } }, // psllw + pand.
{ ISD::SRL, MVT::v64i8, { 2, 12, 5, 6 } }, // psrlw + pand.
@@ -450,6 +468,9 @@ InstructionCost X86TTIImpl::getArithmeticInstrCost(
{ ISD::SREM, MVT::v16i32, { 8 } }, // pmuludq+mul+sub sequence
{ ISD::UDIV, MVT::v16i32, { 5 } }, // pmuludq sequence
{ ISD::UREM, MVT::v16i32, { 7 } }, // pmuludq+mul+sub sequence
+
+ { ISD::UDIV, MVT::v8i64, { 15 } }, // pmuludq-based MULHU sequence
+ { ISD::UREM, MVT::v8i64, { 21 } }, // pmuludq-based MULHU+mul+sub sequence
};
if (Op2Info.isUniform() && Op2Info.isConstant() && ST->hasAVX512())
@@ -491,6 +512,9 @@ InstructionCost X86TTIImpl::getArithmeticInstrCost(
{ ISD::SREM, MVT::v8i32, { 8 } }, // pmuludq+mul+sub sequence
{ ISD::UDIV, MVT::v8i32, { 5 } }, // pmuludq sequence
{ ISD::UREM, MVT::v8i32, { 7 } }, // pmuludq+mul+sub sequence
+
+ { ISD::UDIV, MVT::v4i64, { 15 } }, // pmuludq-based MULHU sequence
+ { ISD::UREM, MVT::v4i64, { 21 } }, // pmuludq-based MULHU+mul+sub sequence
};
if (Op2Info.isUniform() && Op2Info.isConstant() && ST->hasAVX2())
@@ -591,6 +615,23 @@ InstructionCost X86TTIImpl::getArithmeticInstrCost(
if (auto KindCost = Entry->Cost[CostKind])
return LT.first * *KindCost;
+ static const CostKindTblEntry AVX512DQConstCostTable[] = {
+ { ISD::SDIV, MVT::v4i64, { 19 } }, // vpmullq-based MULHS sequence
+ { ISD::SREM, MVT::v4i64, { 21 } }, // vpmullq-based MULHS+mul+sub sequence
+ { ISD::SDIV, MVT::v8i64, { 19 } }, // vpmullq-based MULHS sequence
+ { ISD::SREM, MVT::v8i64, { 21 } }, // vpmullq-based MULHS+mul+sub sequence
+ // The remainder's multiply-back is a single vpmullq with DQ, whereas the
+ // AVX512/AVX2 tables have to charge for another vpmuludq schoolbook.
+ { ISD::UREM, MVT::v4i64, { 24 } }, // MULHU + vpmullq + sub sequence
+ { ISD::UREM, MVT::v8i64, { 24 } }, // MULHU + vpmullq + sub sequence
+ };
+
+ if (Op2Info.isConstant() && ST->hasDQI())
+ if (const auto *Entry =
+ CostTableLookup(AVX512DQConstCostTable, ISD, LT.second))
+ if (auto KindCost = Entry->Cost[CostKind])
+ return LT.first * *KindCost;
+
static const CostKindTblEntry AVX512ConstCostTable[] = {
{ ISD::SDIV, MVT::v64i8, { 28 } }, // 4*ext+4*pmulhw sequence
{ ISD::SREM, MVT::v64i8, { 32 } }, // 4*ext+4*pmulhw+mul+sub sequence
@@ -606,6 +647,9 @@ InstructionCost X86TTIImpl::getArithmeticInstrCost(
{ ISD::SREM, MVT::v16i32, { 17 } }, // vpmuldq+mul+sub sequence
{ ISD::UDIV, MVT::v16i32, { 15 } }, // vpmuludq sequence
{ ISD::UREM, MVT::v16i32, { 17 } }, // vpmuludq+mul+sub sequence
+
+ { ISD::UDIV, MVT::v8i64, { 22 } }, // vpmuludq-based MULHU sequence
+ { ISD::UREM, MVT::v8i64, { 28 } }, // vpmuludq-based MULHU+mul+sub sequence
};
if (Op2Info.isConstant() && ST->hasAVX512())
@@ -629,6 +673,9 @@ InstructionCost X86TTIImpl::getArithmeticInstrCost(
{ ISD::SREM, MVT::v8i32, { 19 } }, // vpmuldq+mul+sub sequence
{ ISD::UDIV, MVT::v8i32, { 15 } }, // vpmuludq sequence
{ ISD::UREM, MVT::v8i32, { 19 } }, // vpmuludq+mul+sub sequence
+
+ { ISD::UDIV, MVT::v4i64, { 22 } }, // vpmuludq-based MULHU sequence
+ { ISD::UREM, MVT::v4i64, { 28 } }, // vpmuludq-based MULHU+mul+sub sequence
};
if (Op2Info.isConstant() && ST->hasAVX2())
diff --git a/llvm/test/Analysis/CostModel/X86/div.ll b/llvm/test/Analysis/CostModel/X86/div.ll
index 3af5b2dd217da..0cba11a867cfa 100644
--- a/llvm/test/Analysis/CostModel/X86/div.ll
+++ b/llvm/test/Analysis/CostModel/X86/div.ll
@@ -10,6 +10,7 @@
; RUN: opt < %s -mtriple=x86_64-apple-macosx10.8.0 -passes="print<cost-model>" 2>&1 -disable-output -cost-kind=all -mcpu=slm | FileCheck %s --check-prefixes=CHECK,SSE,SLM
; RUN: opt < %s -mtriple=x86_64-apple-macosx10.8.0 -passes="print<cost-model>" 2>&1 -disable-output -cost-kind=all -mcpu=goldmont | FileCheck %s --check-prefixes=CHECK,SSE,SSE42
; RUN: opt < %s -mtriple=x86_64-apple-macosx10.8.0 -passes="print<cost-model>" 2>&1 -disable-output -cost-kind=all -mcpu=btver2 | FileCheck %s --check-prefixes=CHECK,AVX1
+; RUN: opt < %s -mtriple=x86_64-apple-macosx10.8.0 -passes="print<cost-model>" 2>&1 -disable-output -cost-kind=all -mattr=+avx512f,+avx512dq | FileCheck %s --check-prefixes=CHECK,AVX512DQ
define i32 @sdiv() {
; CHECK-LABEL: 'sdiv'
@@ -230,6 +231,25 @@ define i32 @sdiv_const() {
; SLM-NEXT: Cost Model: Found costs of RThru:28 CodeSize:4 Lat:4 SizeLat:4 for: %V32i8 = sdiv <32 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
; SLM-NEXT: Cost Model: Found costs of RThru:56 CodeSize:4 Lat:4 SizeLat:4 for: %V64i8 = sdiv <64 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
; SLM-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
+;
+; AVX512DQ-LABEL: 'sdiv_const'
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I64 = sdiv i64 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:40 CodeSize:4 Lat:4 SizeLat:4 for: %V2i64 = sdiv <2 x i64> undef, <i64 6, i64 7>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:19 CodeSize:4 Lat:4 SizeLat:4 for: %V4i64 = sdiv <4 x i64> undef, <i64 4, i64 5, i64 6, i64 7>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:19 CodeSize:4 Lat:4 SizeLat:4 for: %V8i64 = sdiv <8 x i64> undef, <i64 4, i64 5, i64 6, i64 7, i64 8, i64 9, i64 10, i64 11>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I32 = sdiv i32 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V4i32 = sdiv <4 x i32> undef, <i32 4, i32 5, i32 6, i32 7>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V8i32 = sdiv <8 x i32> undef, <i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V16i32 = sdiv <16 x i32> undef, <i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11, i32 12, i32 13, i32 14, i32 15, i32 16, i32 17, i32 18, i32 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I16 = sdiv i16 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:6 CodeSize:4 Lat:4 SizeLat:4 for: %V8i16 = sdiv <8 x i16> undef, <i16 4, i16 5, i16 6, i16 7, i16 8, i16 9, i16 10, i16 11>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:6 CodeSize:4 Lat:4 SizeLat:4 for: %V16i16 = sdiv <16 x i16> undef, <i16 4, i16 5, i16 6, i16 7, i16 8, i16 9, i16 10, i16 11, i16 12, i16 13, i16 14, i16 15, i16 16, i16 17, i16 18, i16 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:12 CodeSize:4 Lat:4 SizeLat:4 for: %V32i16 = sdiv <32 x i16> undef, <i16 4, i16 5, i16 6, i16 7, i16 8, i16 9, i16 10, i16 11, i16 12, i16 13, i16 14, i16 15, i16 16, i16 17, i16 18, i16 19, i16 4, i16 5, i16 6, i16 7, i16 8, i16 9, i16 10, i16 11, i16 12, i16 13, i16 14, i16 15, i16 16, i16 17, i16 18, i16 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I8 = sdiv i8 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:14 CodeSize:4 Lat:4 SizeLat:4 for: %V16i8 = sdiv <16 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:14 CodeSize:4 Lat:4 SizeLat:4 for: %V32i8 = sdiv <32 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:28 CodeSize:4 Lat:4 SizeLat:4 for: %V64i8 = sdiv <64 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
;
%I64 = sdiv i64 undef, 7
%V2i64 = sdiv <2 x i64> undef, <i64 6, i64 7>
@@ -296,8 +316,8 @@ define i32 @udiv_const() {
; AVX2-LABEL: 'udiv_const'
; AVX2-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I64 = udiv i64 undef, 7
; AVX2-NEXT: Cost Model: Found costs of RThru:40 CodeSize:4 Lat:4 SizeLat:4 for: %V2i64 = udiv <2 x i64> undef, <i64 6, i64 7>
-; AVX2-NEXT: Cost Model: Found costs of RThru:80 CodeSize:4 Lat:4 SizeLat:4 for: %V4i64 = udiv <4 x i64> undef, <i64 4, i64 5, i64 6, i64 7>
-; AVX2-NEXT: Cost Model: Found costs of RThru:160 CodeSize:4 Lat:4 SizeLat:4 for: %V8i64 = udiv <8 x i64> undef, <i64 4, i64 5, i64 6, i64 7, i64 8, i64 9, i64 10, i64 11>
+; AVX2-NEXT: Cost Model: Found costs of RThru:22 CodeSize:4 Lat:4 SizeLat:4 for: %V4i64 = udiv <4 x i64> undef, <i64 4, i64 5, i64 6, i64 7>
+; AVX2-NEXT: Cost Model: Found costs of RThru:44 CodeSize:4 Lat:4 SizeLat:4 for: %V8i64 = udiv <8 x i64> undef, <i64 4, i64 5, i64 6, i64 7, i64 8, i64 9, i64 10, i64 11>
; AVX2-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I32 = udiv i32 undef, 7
; AVX2-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V4i32 = udiv <4 x i32> undef, <i32 4, i32 5, i32 6, i32 7>
; AVX2-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V8i32 = udiv <8 x i32> undef, <i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11>
@@ -315,8 +335,8 @@ define i32 @udiv_const() {
; AVX512F-LABEL: 'udiv_const'
; AVX512F-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I64 = udiv i64 undef, 7
; AVX512F-NEXT: Cost Model: Found costs of RThru:40 CodeSize:4 Lat:4 SizeLat:4 for: %V2i64 = udiv <2 x i64> undef, <i64 6, i64 7>
-; AVX512F-NEXT: Cost Model: Found costs of RThru:80 CodeSize:4 Lat:4 SizeLat:4 for: %V4i64 = udiv <4 x i64> undef, <i64 4, i64 5, i64 6, i64 7>
-; AVX512F-NEXT: Cost Model: Found costs of RThru:160 CodeSize:4 Lat:4 SizeLat:4 for: %V8i64 = udiv <8 x i64> undef, <i64 4, i64 5, i64 6, i64 7, i64 8, i64 9, i64 10, i64 11>
+; AVX512F-NEXT: Cost Model: Found costs of RThru:22 CodeSize:4 Lat:4 SizeLat:4 for: %V4i64 = udiv <4 x i64> undef, <i64 4, i64 5, i64 6, i64 7>
+; AVX512F-NEXT: Cost Model: Found costs of RThru:22 CodeSize:4 Lat:4 SizeLat:4 for: %V8i64 = udiv <8 x i64> undef, <i64 4, i64 5, i64 6, i64 7, i64 8, i64 9, i64 10, i64 11>
; AVX512F-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I32 = udiv i32 undef, 7
; AVX512F-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V4i32 = udiv <4 x i32> undef, <i32 4, i32 5, i32 6, i32 7>
; AVX512F-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V8i32 = udiv <8 x i32> undef, <i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11>
@@ -334,8 +354,8 @@ define i32 @udiv_const() {
; AVX512BW-LABEL: 'udiv_const'
; AVX512BW-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I64 = udiv i64 undef, 7
; AVX512BW-NEXT: Cost Model: Found costs of RThru:40 CodeSize:4 Lat:4 SizeLat:4 for: %V2i64 = udiv <2 x i64> undef, <i64 6, i64 7>
-; AVX512BW-NEXT: Cost Model: Found costs of RThru:80 CodeSize:4 Lat:4 SizeLat:4 for: %V4i64 = udiv <4 x i64> undef, <i64 4, i64 5, i64 6, i64 7>
-; AVX512BW-NEXT: Cost Model: Found costs of RThru:160 CodeSize:4 Lat:4 SizeLat:4 for: %V8i64 = udiv <8 x i64> undef, <i64 4, i64 5, i64 6, i64 7, i64 8, i64 9, i64 10, i64 11>
+; AVX512BW-NEXT: Cost Model: Found costs of RThru:22 CodeSize:4 Lat:4 SizeLat:4 for: %V4i64 = udiv <4 x i64> undef, <i64 4, i64 5, i64 6, i64 7>
+; AVX512BW-NEXT: Cost Model: Found costs of RThru:22 CodeSize:4 Lat:4 SizeLat:4 for: %V8i64 = udiv <8 x i64> undef, <i64 4, i64 5, i64 6, i64 7, i64 8, i64 9, i64 10, i64 11>
; AVX512BW-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I32 = udiv i32 undef, 7
; AVX512BW-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V4i32 = udiv <4 x i32> undef, <i32 4, i32 5, i32 6, i32 7>
; AVX512BW-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V8i32 = udiv <8 x i32> undef, <i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11>
@@ -349,6 +369,25 @@ define i32 @udiv_const() {
; AVX512BW-NEXT: Cost Model: Found costs of RThru:14 CodeSize:4 Lat:4 SizeLat:4 for: %V32i8 = udiv <32 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
; AVX512BW-NEXT: Cost Model: Found costs of RThru:14 CodeSize:4 Lat:4 SizeLat:4 for: %V64i8 = udiv <64 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
; AVX512BW-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
+;
+; AVX512DQ-LABEL: 'udiv_const'
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I64 = udiv i64 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:40 CodeSize:4 Lat:4 SizeLat:4 for: %V2i64 = udiv <2 x i64> undef, <i64 6, i64 7>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:22 CodeSize:4 Lat:4 SizeLat:4 for: %V4i64 = udiv <4 x i64> undef, <i64 4, i64 5, i64 6, i64 7>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:22 CodeSize:4 Lat:4 SizeLat:4 for: %V8i64 = udiv <8 x i64> undef, <i64 4, i64 5, i64 6, i64 7, i64 8, i64 9, i64 10, i64 11>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I32 = udiv i32 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V4i32 = udiv <4 x i32> undef, <i32 4, i32 5, i32 6, i32 7>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V8i32 = udiv <8 x i32> undef, <i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V16i32 = udiv <16 x i32> undef, <i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11, i32 12, i32 13, i32 14, i32 15, i32 16, i32 17, i32 18, i32 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I16 = udiv i16 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:6 CodeSize:4 Lat:4 SizeLat:4 for: %V8i16 = udiv <8 x i16> undef, <i16 4, i16 5, i16 6, i16 7, i16 8, i16 9, i16 10, i16 11>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:6 CodeSize:4 Lat:4 SizeLat:4 for: %V16i16 = udiv <16 x i16> undef, <i16 4, i16 5, i16 6, i16 7, i16 8, i16 9, i16 10, i16 11, i16 12, i16 13, i16 14, i16 15, i16 16, i16 17, i16 18, i16 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:12 CodeSize:4 Lat:4 SizeLat:4 for: %V32i16 = udiv <32 x i16> undef, <i16 4, i16 5, i16 6, i16 7, i16 8, i16 9, i16 10, i16 11, i16 12, i16 13, i16 14, i16 15, i16 16, i16 17, i16 18, i16 19, i16 4, i16 5, i16 6, i16 7, i16 8, i16 9, i16 10, i16 11, i16 12, i16 13, i16 14, i16 15, i16 16, i16 17, i16 18, i16 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I8 = udiv i8 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:14 CodeSize:4 Lat:4 SizeLat:4 for: %V16i8 = udiv <16 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:14 CodeSize:4 Lat:4 SizeLat:4 for: %V32i8 = udiv <32 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:28 CodeSize:4 Lat:4 SizeLat:4 for: %V64i8 = udiv <64 x i8> undef, <i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17, i8 18, i8 19>
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
;
%I64 = udiv i64 undef, 7
%V2i64 = udiv <2 x i64> undef, <i64 6, i64 7>
@@ -468,6 +507,25 @@ define i32 @sdiv_uniformconst() {
; AVX512BW-NEXT: Cost Model: Found costs of RThru:14 CodeSize:4 Lat:4 SizeLat:4 for: %V32i8 = sdiv <32 x i8> undef, splat (i8 7)
; AVX512BW-NEXT: Cost Model: Found costs of RThru:14 CodeSize:4 Lat:4 SizeLat:4 for: %V64i8 = sdiv <64 x i8> undef, splat (i8 7)
; AVX512BW-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
+;
+; AVX512DQ-LABEL: 'sdiv_uniformconst'
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I64 = sdiv i64 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:40 CodeSize:4 Lat:4 SizeLat:4 for: %V2i64 = sdiv <2 x i64> undef, splat (i64 7)
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V4i64 = sdiv <4 x i64> undef, splat (i64 7)
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:15 CodeSize:4 Lat:4 SizeLat:4 for: %V8i64 = sdiv <8 x i64> undef, splat (i64 7)
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:1 CodeSize:4 Lat:4 SizeLat:4 for: %I32 = sdiv i32 undef, 7
+; AVX512DQ-NEXT: Cost Model: Found costs of RThru:6 CodeSize:4 Lat:4 SizeLat:4 for: %V4i32 = sdiv <4 x i32> undef, splat (i32 7)
+; AVX512DQ-NEXT: Cost Model: Found costs o...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/208491
More information about the llvm-commits
mailing list