[llvm] [X86][SchedModel] Add masked gather/scatter overrides to Znver4Model (PR #212997)
Sumukh J Bharadwaj via llvm-commits
llvm-commits at lists.llvm.org
Wed Sep 2 02:08:27 PDT 2026
https://github.com/amd-subharad updated https://github.com/llvm/llvm-project/pull/212997
>From 7cf05669ac4f86e67ac7a8a1169d1f2b1545317b Mon Sep 17 00:00:00 2001
From: Sumukh Bharadwaj <Sumukh.Bharadwaj at amd.com>
Date: Wed, 2 Sep 2026 01:15:07 +0530
Subject: [PATCH 1/2] [X86][SchedModel] Add masked gather/scatter overrides to
Znver4Model
Znver4Model had no overrides for the AVX-512 masked gather and scatter
instructions, so they fell back to the default vector load/store write:
one uop, latency 5, reciprocal throughput 0.33. On these parts they are
microcoded sequences costing tens of uops and tens of cycles, so the
default was wrong by more than an order of magnitude.
Add 21 SchedWriteRes overrides covering every EVEX masked gather
(VPGATHER* / VGATHER*) and scatter (VPSCATTER* / VSCATTER*) form, and
bind the AVX2 (VEX) gathers, which were also on the default write and
diverged from it by more than an order of magnitude. All but the 8 x i32
VEX form measure like their EVEX counterparts and reuse those entries;
VPGATHERDDY is genuinely cheaper and gets its own.
Agner and uops.info both show these sequences using the vector pipes
heavily, so every entry charges Zn4FPU0123. Only that resource carries a
fitted number, taken from the measured bottleneck as round(tput * 4);
every other resource carries its plain physical occupancy. Fitting each
resource independently let a non-limiting resource round past the
intended limiter -- the 2 x i64 scatter reported 2.7 cycles against an
intended 2.5 -- and deriving the bottleneck once removes that by
construction.
Throughput, uops and gather latency are measured on Znver4 (Ryzen 5
8645HS). Znver4Model also backs znver5 and znver6, which run these
sequences 25-55% faster and are therefore modeled pessimistically; a
per-part split is left as future work.
Entries are keyed by (#elements, element width, index width). Sharing one
entry between the dword- and qword-index encodings is wrong for six
shapes, worst at the 8 x i64 scatter: 13.94 cycles with a qword index
against 11.05 with a dword one, at equal uop counts over identical
address sets. The four 2-element shapes and both 4-element gathers agree
to within 0.03 cycles and keep a shared entry. The 2-element shapes are
force-scalarised by the auto-vectoriser but remain reachable through the
intrinsics, so they are measured rather than extrapolated.
Gather latency is measured from a dependency chain that feeds the loaded
data back into the index register, with the feedback op's own latency
subtracted. Scatter latency remains estimated, as there is no dependent
consumer to time.
The instruction destroys its own mask and so cannot be timed alone;
substituting an equal-cost kxnorw for the kmovd isolates the reload,
which only changes the total for the 2-element gathers (5.00 vs 3.98
cycles), so only those have it subtracted. Sub-cycle accounting stops
there: identical code in two processes agrees to a p90 of 0.24 cycles,
but the same instructions in a different schedule move by a p90 of 1.35,
so the table is accurate to about a cycle regardless.
llvm-mca now tracks hardware within 1.3% on every scatter shape and
within 1.1% on every gather of four elements or more. Add a CodeGen test
so the values are exercised by something other than the llvm-mca
resource tests, since these writes also feed the machine scheduler.
---
llvm/lib/Target/X86/X86ScheduleZnver4.td | 137 ++++++++++++++++++
.../X86/znver4-gather-scatter-schedule.ll | 36 +++++
.../llvm-mca/X86/Znver4/resources-avx2.s | 66 ++++-----
.../llvm-mca/X86/Znver4/resources-avx512.s | 66 ++++-----
.../llvm-mca/X86/Znver4/resources-avx512vl.s | 130 ++++++++---------
5 files changed, 304 insertions(+), 131 deletions(-)
create mode 100644 llvm/test/CodeGen/X86/znver4-gather-scatter-schedule.ll
diff --git a/llvm/lib/Target/X86/X86ScheduleZnver4.td b/llvm/lib/Target/X86/X86ScheduleZnver4.td
index 34e57f634a1e6..e3711a0701977 100644
--- a/llvm/lib/Target/X86/X86ScheduleZnver4.td
+++ b/llvm/lib/Target/X86/X86ScheduleZnver4.td
@@ -509,6 +509,143 @@ defm : Zn4WriteResInt<WriteLoad, [Zn4AGU012, Zn4Load], !add(Znver4Model.LoadLate
// Does not cost anything by itself, only has latency, matching that of the WriteLoad,
defm : Zn4WriteResInt<WriteVecMaskedGatherWriteback, [], !add(Znver4Model.LoadLatency, 1), [], 0>;
+// AVX-512 masked GATHER / SCATTER. Microcoded, so entries are keyed by
+// (#elements, element width, index width); the two index widths share an entry
+// only where they measure alike. Zn4FPU0123 carries the fitted bottleneck,
+// round(tput * 4), and every other resource its physical occupancy, so a
+// retune edits the Zn4FPU0123 number. Measured on Znver4 and accurate to about
+// a cycle; scatter latency is estimated rather than timed. Znver4Model also
+// backs znver5 and znver6, which run 25-55% faster and are pessimistic here.
+// The commit message describes how the numbers were measured.
+def Zn4WriteVPGATHERQDZ128 : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [2, 2, 2, 16]; let Latency = 13; let NumMicroOps = 18;
+}
+def : InstRW<[Zn4WriteVPGATHERQDZ128, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQDZ128rm, VGATHERQPSZ128rm)>;
+def Zn4WriteVPGATHERDDZ128 : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [4, 4, 4, 20]; let Latency = 15; let NumMicroOps = 24;
+}
+def : InstRW<[Zn4WriteVPGATHERDDZ128, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERDDZ128rm, VGATHERDPSZ128rm,
+ VPGATHERQDZ256rm, VGATHERQPSZ256rm)>;
+def Zn4WriteVPGATHERDDZ256 : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [8, 8, 8, 37]; let Latency = 20; let NumMicroOps = 41;
+}
+def : InstRW<[Zn4WriteVPGATHERDDZ256, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERDDZ256rm, VGATHERDPSZ256rm)>;
+def Zn4WriteVPGATHERQDZ : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [8, 8, 8, 39]; let Latency = 24; let NumMicroOps = 46;
+}
+def : InstRW<[Zn4WriteVPGATHERQDZ, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQDZrm, VGATHERQPSZrm)>;
+def Zn4WriteVPGATHERDDZ : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [16, 16, 16, 70]; let Latency = 32; let NumMicroOps = 81;
+}
+def : InstRW<[Zn4WriteVPGATHERDDZ, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERDDZrm, VGATHERDPSZrm)>;
+def Zn4WriteVPGATHERQQZ128 : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [2, 2, 2, 16]; let Latency = 12; let NumMicroOps = 17;
+}
+def : InstRW<[Zn4WriteVPGATHERQQZ128, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQQZ128rm, VGATHERQPDZ128rm,
+ VPGATHERDQZ128rm, VGATHERDPDZ128rm)>;
+def Zn4WriteVPGATHERQQZ256 : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [4, 4, 4, 20]; let Latency = 16; let NumMicroOps = 24;
+}
+def : InstRW<[Zn4WriteVPGATHERQQZ256, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQQZ256rm, VGATHERQPDZ256rm,
+ VPGATHERDQZ256rm, VGATHERDPDZ256rm)>;
+def Zn4WriteVPGATHERQQZ : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [8, 8, 8, 39]; let Latency = 25; let NumMicroOps = 48;
+}
+def : InstRW<[Zn4WriteVPGATHERQQZ, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQQZrm, VGATHERQPDZrm)>;
+def Zn4WriteVPGATHERDQZ : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [8, 8, 8, 38]; let Latency = 22; let NumMicroOps = 46;
+}
+def : InstRW<[Zn4WriteVPGATHERDQZ, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERDQZrm, VGATHERDPDZrm)>;
+
+// AVX2 (VEX) gathers. AVX2 has no scatter and no VEX form exceeds 8 x i32. All
+// but 8 x i32 measure like the EVEX entry for their shape and reuse it; 8 x i32
+// is genuinely cheaper (8.35 vs 9.19 cyc) and gets its own entry. An InstRW
+// replaces the whole Sched<> list, so each one must restate the mask-writeback
+// def that X86InstrSSE.td attaches; it is the EVEX one, an AVX-512-latency
+// write, and a VEX-specific one is future work.
+def : InstRW<[Zn4WriteVPGATHERQDZ128, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQDrm, VGATHERQPSrm)>;
+def : InstRW<[Zn4WriteVPGATHERDDZ128, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERDDrm, VGATHERDPSrm,
+ VPGATHERQDYrm, VGATHERQPSYrm)>;
+def Zn4WriteVPGATHERDDY : SchedWriteRes<[Zn4AGU012, Zn4Load, Zn4FPLd01, Zn4FPU0123]> {
+ let ReleaseAtCycles = [8, 8, 8, 33]; let Latency = 21; let NumMicroOps = 42;
+}
+def : InstRW<[Zn4WriteVPGATHERDDY, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERDDYrm, VGATHERDPSYrm)>;
+def : InstRW<[Zn4WriteVPGATHERQQZ128, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQQrm, VGATHERQPDrm,
+ VPGATHERDQrm, VGATHERDPDrm)>;
+def : InstRW<[Zn4WriteVPGATHERQQZ256, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQQYrm, VGATHERQPDYrm,
+ VPGATHERDQYrm, VGATHERDPDYrm)>;
+
+def Zn4WriteVPSCATTERQDZ128 : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [2, 2, 2, 16]; let Latency = 6; let NumMicroOps = 17;
+}
+def : InstRW<[Zn4WriteVPSCATTERQDZ128],
+ (instrs VPSCATTERQDZ128mr, VSCATTERQPSZ128mr)>;
+def Zn4WriteVPSCATTERDDZ128 : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [4, 4, 4, 33]; let Latency = 9; let NumMicroOps = 27;
+}
+def : InstRW<[Zn4WriteVPSCATTERDDZ128],
+ (instrs VPSCATTERDDZ128mr, VSCATTERDPSZ128mr)>;
+def Zn4WriteVPSCATTERQDZ256 : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [4, 4, 4, 26]; let Latency = 9; let NumMicroOps = 27;
+}
+def : InstRW<[Zn4WriteVPSCATTERQDZ256],
+ (instrs VPSCATTERQDZ256mr, VSCATTERQPSZ256mr)>;
+def Zn4WriteVPSCATTERDDZ256 : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [8, 8, 8, 49]; let Latency = 14; let NumMicroOps = 48;
+}
+def : InstRW<[Zn4WriteVPSCATTERDDZ256],
+ (instrs VPSCATTERDDZ256mr, VSCATTERDPSZ256mr)>;
+def Zn4WriteVPSCATTERQDZ : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [8, 8, 8, 46]; let Latency = 14; let NumMicroOps = 48;
+}
+def : InstRW<[Zn4WriteVPSCATTERQDZ],
+ (instrs VPSCATTERQDZmr, VSCATTERQPSZmr)>;
+def Zn4WriteVPSCATTERDDZ : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [16, 16, 16, 103]; let Latency = 24; let NumMicroOps = 89;
+}
+def : InstRW<[Zn4WriteVPSCATTERDDZ],
+ (instrs VPSCATTERDDZmr, VSCATTERDPSZmr)>;
+def Zn4WriteVPSCATTERQQZ128 : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [2, 2, 2, 16]; let Latency = 6; let NumMicroOps = 17;
+}
+def : InstRW<[Zn4WriteVPSCATTERQQZ128],
+ (instrs VPSCATTERQQZ128mr, VSCATTERQPDZ128mr,
+ VPSCATTERDQZ128mr, VSCATTERDPDZ128mr)>;
+def Zn4WriteVPSCATTERQQZ256 : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [4, 4, 4, 26]; let Latency = 9; let NumMicroOps = 27;
+}
+def : InstRW<[Zn4WriteVPSCATTERQQZ256],
+ (instrs VPSCATTERQQZ256mr, VSCATTERQPDZ256mr)>;
+def Zn4WriteVPSCATTERDQZ256 : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [4, 4, 4, 24]; let Latency = 9; let NumMicroOps = 28;
+}
+def : InstRW<[Zn4WriteVPSCATTERDQZ256],
+ (instrs VPSCATTERDQZ256mr, VSCATTERDPDZ256mr)>;
+def Zn4WriteVPSCATTERQQZ : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [8, 8, 8, 56]; let Latency = 14; let NumMicroOps = 48;
+}
+def : InstRW<[Zn4WriteVPSCATTERQQZ],
+ (instrs VPSCATTERQQZmr, VSCATTERQPDZmr)>;
+def Zn4WriteVPSCATTERDQZ : SchedWriteRes<[Zn4AGU012, Zn4Store, Zn4FPSt, Zn4FPU0123]> {
+ let ReleaseAtCycles = [8, 8, 8, 44]; let Latency = 14; let NumMicroOps = 48;
+}
+def : InstRW<[Zn4WriteVPSCATTERDQZ],
+ (instrs VPSCATTERDQZmr, VSCATTERDPDZmr)>;
+
def Zn4WriteMOVSlow : SchedWriteRes<[Zn4AGU012, Zn4Load]> {
let Latency = !add(Znver4Model.LoadLatency, 1);
let ReleaseAtCycles = [3, 1];
diff --git a/llvm/test/CodeGen/X86/znver4-gather-scatter-schedule.ll b/llvm/test/CodeGen/X86/znver4-gather-scatter-schedule.ll
new file mode 100644
index 0000000000000..8a3f8d2ba15dc
--- /dev/null
+++ b/llvm/test/CodeGen/X86/znver4-gather-scatter-schedule.ll
@@ -0,0 +1,36 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=znver4 | FileCheck %s
+
+; The znver4 schedule model carries per-shape masked gather/scatter overrides
+; (see X86ScheduleZnver4.td). No other znver-tuned CodeGen test exercises these
+; instructions, so this pins which gather/scatter form znver4/znver5/znver6
+; select. It checks lowering only; the latency, throughput and uop counts of the
+; overrides are pinned by the llvm-mca tests under
+; test/tools/llvm-mca/X86/Znver4.
+
+declare <8 x i32> @llvm.masked.gather.v8i32.v8p0(<8 x ptr>, i32, <8 x i1>, <8 x i32>)
+declare void @llvm.masked.scatter.v8i32.v8p0(<8 x i32>, <8 x ptr>, i32, <8 x i1>)
+
+define <8 x i32> @gather_v8i32(<8 x ptr> %ptrs, <8 x i1> %mask, <8 x i32> %passthru) {
+; CHECK-LABEL: gather_v8i32:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vpsllw $15, %xmm1, %xmm1
+; CHECK-NEXT: vpmovw2m %xmm1, %k1
+; CHECK-NEXT: vpgatherqd (,%zmm0), %ymm2 {%k1}
+; CHECK-NEXT: vmovdqa %ymm2, %ymm0
+; CHECK-NEXT: retq
+ %v = call <8 x i32> @llvm.masked.gather.v8i32.v8p0(<8 x ptr> %ptrs, i32 4, <8 x i1> %mask, <8 x i32> %passthru)
+ ret <8 x i32> %v
+}
+
+define void @scatter_v8i32(<8 x ptr> %ptrs, <8 x i1> %mask, <8 x i32> %val) {
+; CHECK-LABEL: scatter_v8i32:
+; CHECK: # %bb.0:
+; CHECK-NEXT: vpsllw $15, %xmm1, %xmm1
+; CHECK-NEXT: vpmovw2m %xmm1, %k1
+; CHECK-NEXT: vpscatterqd %ymm2, (,%zmm0) {%k1}
+; CHECK-NEXT: vzeroupper
+; CHECK-NEXT: retq
+ call void @llvm.masked.scatter.v8i32.v8p0(<8 x i32> %val, <8 x ptr> %ptrs, i32 4, <8 x i1> %mask)
+ ret void
+}
diff --git a/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx2.s b/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx2.s
index 6c8fac4566498..69f8a4961dc92 100644
--- a/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx2.s
+++ b/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx2.s
@@ -465,14 +465,14 @@ vpxor (%rax), %ymm1, %ymm2
# CHECK-NEXT: 1 2 1.00 vbroadcastss %xmm0, %ymm0
# CHECK-NEXT: 1 4 1.00 vextracti128 $1, %ymm0, %xmm2
# CHECK-NEXT: 2 11 1.00 * vextracti128 $1, %ymm0, (%rax)
-# CHECK-NEXT: 1 5 0.33 * vgatherdpd %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 1 5 0.33 * vgatherdpd %ymm0, (%rax,%xmm1,2), %ymm2
-# CHECK-NEXT: 1 5 0.33 * vgatherdps %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 1 5 0.33 * vgatherdps %ymm0, (%rax,%ymm1,2), %ymm2
-# CHECK-NEXT: 1 5 0.33 * vgatherqpd %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 1 5 0.33 * vgatherqpd %ymm0, (%rax,%ymm1,2), %ymm2
-# CHECK-NEXT: 1 5 0.33 * vgatherqps %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 1 5 0.33 * vgatherqps %xmm0, (%rax,%ymm1,2), %xmm2
+# CHECK-NEXT: 17 12 4.00 * vgatherdpd %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 24 16 5.00 * vgatherdpd %ymm0, (%rax,%xmm1,2), %ymm2
+# CHECK-NEXT: 24 15 5.00 * vgatherdps %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 42 21 8.25 * vgatherdps %ymm0, (%rax,%ymm1,2), %ymm2
+# CHECK-NEXT: 17 12 4.00 * vgatherqpd %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 24 16 5.00 * vgatherqpd %ymm0, (%rax,%ymm1,2), %ymm2
+# CHECK-NEXT: 18 13 4.00 * vgatherqps %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 24 15 5.00 * vgatherqps %xmm0, (%rax,%ymm1,2), %xmm2
# CHECK-NEXT: 1 1 1.00 vinserti128 $1, %xmm0, %ymm1, %ymm2
# CHECK-NEXT: 1 8 1.00 * vinserti128 $1, (%rax), %ymm1, %ymm2
# CHECK-NEXT: 1 8 0.50 * vmovntdqa (%rax), %ymm0
@@ -568,14 +568,14 @@ vpxor (%rax), %ymm1, %ymm2
# CHECK-NEXT: 1 11 1.00 * vpermps (%rax), %ymm1, %ymm2
# CHECK-NEXT: 1 4 1.00 vpermq $1, %ymm0, %ymm2
# CHECK-NEXT: 1 11 1.00 * vpermq $1, (%rax), %ymm2
-# CHECK-NEXT: 1 5 0.33 * vpgatherdd %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 1 5 0.33 * vpgatherdd %ymm0, (%rax,%ymm1,2), %ymm2
-# CHECK-NEXT: 1 5 0.33 * vpgatherdq %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 1 5 0.33 * vpgatherdq %ymm0, (%rax,%xmm1,2), %ymm2
-# CHECK-NEXT: 1 5 0.33 * vpgatherqd %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 1 5 0.33 * vpgatherqd %xmm0, (%rax,%ymm1,2), %xmm2
-# CHECK-NEXT: 1 5 0.33 * vpgatherqq %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 1 5 0.33 * vpgatherqq %ymm0, (%rax,%ymm1,2), %ymm2
+# CHECK-NEXT: 24 15 5.00 * vpgatherdd %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 42 21 8.25 * vpgatherdd %ymm0, (%rax,%ymm1,2), %ymm2
+# CHECK-NEXT: 17 12 4.00 * vpgatherdq %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 24 16 5.00 * vpgatherdq %ymm0, (%rax,%xmm1,2), %ymm2
+# CHECK-NEXT: 18 13 4.00 * vpgatherqd %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 24 15 5.00 * vpgatherqd %xmm0, (%rax,%ymm1,2), %xmm2
+# CHECK-NEXT: 17 12 4.00 * vpgatherqq %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 24 16 5.00 * vpgatherqq %ymm0, (%rax,%ymm1,2), %ymm2
# CHECK-NEXT: 3 3 3.00 vphaddd %ymm0, %ymm1, %ymm2
# CHECK-NEXT: 4 10 3.00 * vphaddd (%rax), %ymm1, %ymm2
# CHECK-NEXT: 3 3 3.00 vphaddsw %ymm0, %ymm1, %ymm2
@@ -789,7 +789,7 @@ vpxor (%rax), %ymm1, %ymm2
# CHECK: Resource pressure per iteration:
# CHECK-NEXT: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12.0] [12.1] [13] [14.0] [14.1] [14.2] [15.0] [15.1] [15.2] [16.0] [16.1]
-# CHECK-NEXT: 6.67 6.67 6.67 - - - - - 93.75 128.75 92.25 36.25 80.50 80.50 29.00 52.33 52.33 52.33 50.67 50.67 50.67 2.50 2.50
+# CHECK-NEXT: 21.33 21.33 21.33 - - - - - 174.25 209.25 172.75 116.75 110.50 110.50 29.00 67.00 67.00 67.00 65.33 65.33 65.33 2.50 2.50
# CHECK: Resource pressure by instruction:
# CHECK-NEXT: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12.0] [12.1] [13] [14.0] [14.1] [14.2] [15.0] [15.1] [15.2] [16.0] [16.1] Instructions:
@@ -798,14 +798,14 @@ vpxor (%rax), %ymm1, %ymm2
# CHECK-NEXT: - - - - - - - - - 1.00 - - - - - - - - - - - - - vbroadcastss %xmm0, %ymm0
# CHECK-NEXT: - - - - - - - - 1.00 - - - - - - - - - - - - - - vextracti128 $1, %ymm0, %xmm2
# CHECK-NEXT: - - - - - - - - 1.00 - - - 0.50 0.50 1.00 0.33 0.33 0.33 - - - 0.50 0.50 vextracti128 $1, %ymm0, (%rax)
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdpd %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdpd %ymm0, (%rax,%xmm1,2), %ymm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdps %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdps %ymm0, (%rax,%ymm1,2), %ymm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqpd %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqpd %ymm0, (%rax,%ymm1,2), %ymm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqps %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqps %xmm0, (%rax,%ymm1,2), %xmm2
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vgatherdpd %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vgatherdpd %ymm0, (%rax,%xmm1,2), %ymm2
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vgatherdps %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 8.25 8.25 8.25 8.25 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vgatherdps %ymm0, (%rax,%ymm1,2), %ymm2
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vgatherqpd %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vgatherqpd %ymm0, (%rax,%ymm1,2), %ymm2
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vgatherqps %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vgatherqps %xmm0, (%rax,%ymm1,2), %xmm2
# CHECK-NEXT: - - - - - - - - - 1.00 - - - - - - - - - - - - - vinserti128 $1, %xmm0, %ymm1, %ymm2
# CHECK-NEXT: - - - - - - - - - 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vinserti128 $1, (%rax), %ymm1, %ymm2
# CHECK-NEXT: - - - - - - - - - - - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vmovntdqa (%rax), %ymm0
@@ -901,14 +901,14 @@ vpxor (%rax), %ymm1, %ymm2
# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - 1.00 - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpermps (%rax), %ymm1, %ymm2
# CHECK-NEXT: - - - - - - - - - 1.00 - - - - - - - - - - - - - vpermq $1, %ymm0, %ymm2
# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - 1.00 - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpermq $1, (%rax), %ymm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdd %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdd %ymm0, (%rax,%ymm1,2), %ymm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdq %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdq %ymm0, (%rax,%xmm1,2), %ymm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqd %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqd %xmm0, (%rax,%ymm1,2), %xmm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqq %xmm0, (%rax,%xmm1,2), %xmm2
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqq %ymm0, (%rax,%ymm1,2), %ymm2
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vpgatherdd %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 8.25 8.25 8.25 8.25 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vpgatherdd %ymm0, (%rax,%ymm1,2), %ymm2
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vpgatherdq %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vpgatherdq %ymm0, (%rax,%xmm1,2), %ymm2
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vpgatherqd %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vpgatherqd %xmm0, (%rax,%ymm1,2), %xmm2
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vpgatherqq %xmm0, (%rax,%xmm1,2), %xmm2
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vpgatherqq %ymm0, (%rax,%ymm1,2), %ymm2
# CHECK-NEXT: - - - - - - - - 3.00 - - - - - - - - - - - - - - vphaddd %ymm0, %ymm1, %ymm2
# CHECK-NEXT: - - - - - - - - 3.00 - - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vphaddd (%rax), %ymm1, %ymm2
# CHECK-NEXT: - - - - - - - - 3.00 - - - - - - - - - - - - - - vphaddsw %ymm0, %ymm1, %ymm2
diff --git a/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s b/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s
index f1712aca08311..566a1a9e48126 100644
--- a/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s
+++ b/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s
@@ -1495,10 +1495,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: 1 4 1.00 vfmadd231ps %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 1 11 1.00 * vfmadd231ps (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 1 11 1.00 * vfmadd231ps (%rax){1to16}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 1 5 0.33 * vgatherdpd (%rax,%ymm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherdps (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherqpd (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherqps (%rax,%zmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 46 22 9.50 * vgatherdpd (%rax,%ymm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 81 32 17.50 * vgatherdps (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 48 25 9.75 * vgatherqpd (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 46 24 9.75 * vgatherqps (%rax,%zmm1,2), %ymm2 {%k1}
# CHECK-NEXT: 1 2 1.00 vmaxpd %zmm16, %zmm17, %zmm19
# CHECK-NEXT: 1 9 1.00 * vmaxpd (%rax), %zmm17, %zmm19
# CHECK-NEXT: 1 9 1.00 * vmaxpd (%rax){1to8}, %zmm17, %zmm19
@@ -1732,10 +1732,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: 1 1 0.50 vpcmpequq %zmm0, %zmm1, %k2 {%k3}
# CHECK-NEXT: 1 8 0.50 * vpcmpequq (%rax), %zmm1, %k2 {%k3}
# CHECK-NEXT: 1 8 0.50 * vpcmpequq (%rax){1to8}, %zmm1, %k2 {%k3}
-# CHECK-NEXT: 1 5 0.33 * vpgatherdq (%rax,%ymm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherdd (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherqq (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherqd (%rax,%zmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 46 22 9.50 * vpgatherdq (%rax,%ymm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 81 32 17.50 * vpgatherdd (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 48 25 9.75 * vpgatherqq (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 46 24 9.75 * vpgatherqd (%rax,%zmm1,2), %ymm2 {%k1}
# CHECK-NEXT: 1 5 1.00 vpmovdb %zmm19, %xmm16
# CHECK-NEXT: 1 11 1.50 * vpmovdb %zmm19, (%rax)
# CHECK-NEXT: 1 5 1.00 vpmovdb %zmm19, %xmm16 {%k1}
@@ -1970,10 +1970,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: 2 1 0.50 vpermq %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 2 8 0.50 * vpermq (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 2 8 0.50 * vpermq (%rax){1to8}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 1 1 1.00 * vpscatterdd %zmm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterdq %zmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterqd %ymm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterqq %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 89 24 25.75 * vpscatterdd %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 48 14 11.00 * vpscatterdq %zmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 48 14 11.50 * vpscatterqd %ymm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 48 14 14.00 * vpscatterqq %zmm1, (%rdx,%zmm0,4) {%k1}
# CHECK-NEXT: 1 1 1.00 vpshufd $0, %zmm16, %zmm19
# CHECK-NEXT: 1 8 1.00 * vpshufd $0, (%rax), %zmm19
# CHECK-NEXT: 1 8 1.00 * vpshufd $0, (%rax){1to16}, %zmm19
@@ -2037,10 +2037,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: 1 1 1.00 vpunpcklqdq %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 1 8 1.00 * vpunpcklqdq (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 1 8 1.00 * vpunpcklqdq (%rax){1to8}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 1 1 1.00 * vscatterdps %zmm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterdpd %zmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterqps %ymm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterqpd %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 89 24 25.75 * vscatterdps %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 48 14 11.00 * vscatterdpd %zmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 48 14 11.50 * vscatterqps %ymm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 48 14 14.00 * vscatterqpd %zmm1, (%rdx,%zmm0,4) {%k1}
# CHECK-NEXT: 1 2 1.00 vshuff32x4 $0, %zmm16, %zmm17, %zmm19
# CHECK-NEXT: 3 9 1.00 * vshuff32x4 $0, (%rax), %zmm17, %zmm19
# CHECK-NEXT: 3 9 1.00 * vshuff32x4 $0, (%rax){1to16}, %zmm17, %zmm19
@@ -2233,7 +2233,7 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK: Resource pressure per iteration:
# CHECK-NEXT: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12.0] [12.1] [13] [14.0] [14.1] [14.2] [15.0] [15.1] [15.2] [16.0] [16.1]
-# CHECK-NEXT: 5.33 5.33 5.33 - - - - - 221.00 1120.50 678.00 352.50 312.50 312.50 17.00 215.67 215.67 215.67 204.67 204.67 204.67 16.50 16.50
+# CHECK-NEXT: 53.33 53.33 53.33 - - - - - 438.50 1338.00 895.50 570.00 392.50 392.50 97.00 261.00 261.00 261.00 228.67 228.67 228.67 48.50 48.50
# CHECK: Resource pressure by instruction:
# CHECK-NEXT: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12.0] [12.1] [13] [14.0] [14.1] [14.2] [15.0] [15.1] [15.2] [16.0] [16.1] Instructions:
@@ -2552,10 +2552,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - - - - - - - - - - - - vfmadd231ps %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vfmadd231ps (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vfmadd231ps (%rax){1to16}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdpd (%rax,%ymm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdps (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqpd (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqps (%rax,%zmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 9.50 9.50 9.50 9.50 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vgatherdpd (%rax,%ymm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 5.33 5.33 5.33 - - - - - 17.50 17.50 17.50 17.50 8.00 8.00 - 5.33 5.33 5.33 5.33 5.33 5.33 - - vgatherdps (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 9.75 9.75 9.75 9.75 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vgatherqpd (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 9.75 9.75 9.75 9.75 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vgatherqps (%rax,%zmm1,2), %ymm2 {%k1}
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - - - - - - - - - - - - vmaxpd %zmm16, %zmm17, %zmm19
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vmaxpd (%rax), %zmm17, %zmm19
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vmaxpd (%rax){1to8}, %zmm17, %zmm19
@@ -2789,10 +2789,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - 0.50 0.50 0.50 0.50 - - - - - - - - - - - vpcmpequq %zmm0, %zmm1, %k2 {%k3}
# CHECK-NEXT: - - - - - - - - 0.50 0.50 0.50 0.50 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpcmpequq (%rax), %zmm1, %k2 {%k3}
# CHECK-NEXT: - - - - - - - - 0.50 0.50 0.50 0.50 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpcmpequq (%rax){1to8}, %zmm1, %k2 {%k3}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdq (%rax,%ymm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdd (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqq (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqd (%rax,%zmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 9.50 9.50 9.50 9.50 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vpgatherdq (%rax,%ymm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 5.33 5.33 5.33 - - - - - 17.50 17.50 17.50 17.50 8.00 8.00 - 5.33 5.33 5.33 5.33 5.33 5.33 - - vpgatherdd (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 9.75 9.75 9.75 9.75 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vpgatherqq (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 9.75 9.75 9.75 9.75 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vpgatherqd (%rax,%zmm1,2), %ymm2 {%k1}
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - - - - - - - - - - - - vpmovdb %zmm19, %xmm16
# CHECK-NEXT: - - - - - - - - - 1.50 1.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpmovdb %zmm19, (%rax)
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - - - - - - - - - - - - vpmovdb %zmm19, %xmm16 {%k1}
@@ -3027,10 +3027,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - - - - - - - - - - - - vpermq %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpermq (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpermq (%rax){1to8}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterdd %zmm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterdq %zmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterqd %ymm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterqq %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 5.33 5.33 5.33 - - - - - 25.75 25.75 25.75 25.75 8.00 8.00 16.00 5.33 5.33 5.33 - - - 8.00 8.00 vpscatterdd %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 11.00 11.00 11.00 11.00 4.00 4.00 8.00 2.67 2.67 2.67 - - - 4.00 4.00 vpscatterdq %zmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 11.50 11.50 11.50 11.50 4.00 4.00 8.00 2.67 2.67 2.67 - - - 4.00 4.00 vpscatterqd %ymm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 14.00 14.00 14.00 14.00 4.00 4.00 8.00 2.67 2.67 2.67 - - - 4.00 4.00 vpscatterqq %zmm1, (%rdx,%zmm0,4) {%k1}
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - - - - - - - - - - - - vpshufd $0, %zmm16, %zmm19
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpshufd $0, (%rax), %zmm19
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpshufd $0, (%rax){1to16}, %zmm19
@@ -3094,10 +3094,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - - - - - - - - - - - - vpunpcklqdq %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpunpcklqdq (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpunpcklqdq (%rax){1to8}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterdps %zmm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterdpd %zmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterqps %ymm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterqpd %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 5.33 5.33 5.33 - - - - - 25.75 25.75 25.75 25.75 8.00 8.00 16.00 5.33 5.33 5.33 - - - 8.00 8.00 vscatterdps %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 11.00 11.00 11.00 11.00 4.00 4.00 8.00 2.67 2.67 2.67 - - - 4.00 4.00 vscatterdpd %zmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 11.50 11.50 11.50 11.50 4.00 4.00 8.00 2.67 2.67 2.67 - - - 4.00 4.00 vscatterqps %ymm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 14.00 14.00 14.00 14.00 4.00 4.00 8.00 2.67 2.67 2.67 - - - 4.00 4.00 vscatterqpd %zmm1, (%rdx,%zmm0,4) {%k1}
# CHECK-NEXT: - - - - - - - - - 1.00 - - - - - - - - - - - - - vshuff32x4 $0, %zmm16, %zmm17, %zmm19
# CHECK-NEXT: - - - - - - - - - 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vshuff32x4 $0, (%rax), %zmm17, %zmm19
# CHECK-NEXT: - - - - - - - - - 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vshuff32x4 $0, (%rax){1to16}, %zmm17, %zmm19
diff --git a/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512vl.s b/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512vl.s
index ead609e33da4d..68623e7e27ecc 100644
--- a/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512vl.s
+++ b/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512vl.s
@@ -2392,14 +2392,14 @@ vunpcklps (%rax){1to8}, %ymm17, %ymm19 {z}{k1}
# CHECK-NEXT: 1 4 0.50 vfmadd231ps %ymm16, %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: 1 11 0.50 * vfmadd231ps (%rax), %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: 1 11 0.50 * vfmadd231ps (%rax){1to8}, %ymm17, %ymm19 {%k1} {z}
-# CHECK-NEXT: 1 5 0.33 * vgatherdpd (%rax,%xmm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherdps (%rax,%ymm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherqpd (%rax,%ymm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherqps (%rax,%ymm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherdpd (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherdps (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherqpd (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherqps (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 24 16 5.00 * vgatherdpd (%rax,%xmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 41 20 9.25 * vgatherdps (%rax,%ymm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 24 16 5.00 * vgatherqpd (%rax,%ymm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 24 15 5.00 * vgatherqps (%rax,%ymm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 17 12 4.00 * vgatherdpd (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 24 15 5.00 * vgatherdps (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 17 12 4.00 * vgatherqpd (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 18 13 4.00 * vgatherqps (%rax,%xmm1,2), %xmm2 {%k1}
# CHECK-NEXT: 1 2 1.00 vmaxpd %xmm16, %xmm17, %xmm19
# CHECK-NEXT: 1 9 0.50 * vmaxpd (%rax), %xmm17, %xmm19
# CHECK-NEXT: 1 9 0.50 * vmaxpd (%rax){1to2}, %xmm17, %xmm19
@@ -2956,14 +2956,14 @@ vunpcklps (%rax){1to8}, %ymm17, %ymm19 {z}{k1}
# CHECK-NEXT: 1 4 1.00 vpermq %ymm16, %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: 2 8 0.50 * vpermq (%rax), %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: 2 8 0.50 * vpermq (%rax){1to4}, %ymm17, %ymm19 {%k1} {z}
-# CHECK-NEXT: 1 5 0.33 * vpgatherdq (%rax,%xmm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherdd (%rax,%ymm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherqq (%rax,%ymm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherqd (%rax,%ymm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherdq (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherdd (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherqq (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherqd (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 24 16 5.00 * vpgatherdq (%rax,%xmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 41 20 9.25 * vpgatherdd (%rax,%ymm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 24 16 5.00 * vpgatherqq (%rax,%ymm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 24 15 5.00 * vpgatherqd (%rax,%ymm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 17 12 4.00 * vpgatherdq (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 24 15 5.00 * vpgatherdd (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 17 12 4.00 * vpgatherqq (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 18 13 4.00 * vpgatherqd (%rax,%xmm1,2), %xmm2 {%k1}
# CHECK-NEXT: 1 2 0.50 vpmovdb %xmm19, %xmm16
# CHECK-NEXT: 1 11 1.50 * vpmovdb %xmm19, (%rax)
# CHECK-NEXT: 1 2 0.50 vpmovdb %xmm19, %xmm16 {%k1}
@@ -3252,14 +3252,14 @@ vunpcklps (%rax){1to8}, %ymm17, %ymm19 {z}{k1}
# CHECK-NEXT: 1 3 0.50 vpmulld %ymm16, %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: 1 10 0.50 * vpmulld (%rax), %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: 1 10 0.50 * vpmulld (%rax){1to8}, %ymm17, %ymm19 {%k1} {z}
-# CHECK-NEXT: 1 1 1.00 * vpscatterdd %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterdq %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterqd %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterqq %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterdd %ymm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterdq %ymm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterqd %xmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterqq %ymm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 27 9 8.25 * vpscatterdd %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 17 6 4.00 * vpscatterdq %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 17 6 4.00 * vpscatterqd %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 17 6 4.00 * vpscatterqq %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 48 14 12.25 * vpscatterdd %ymm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 28 9 6.00 * vpscatterdq %ymm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 27 9 6.50 * vpscatterqd %xmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 27 9 6.50 * vpscatterqq %ymm1, (%rdx,%ymm0,4) {%k1}
# CHECK-NEXT: 1 1 0.50 vpshufd $0, %xmm16, %xmm19
# CHECK-NEXT: 1 8 0.50 * vpshufd $0, (%rax), %xmm19
# CHECK-NEXT: 1 8 0.50 * vpshufd $0, (%rax){1to4}, %xmm19
@@ -3398,14 +3398,14 @@ vunpcklps (%rax){1to8}, %ymm17, %ymm19 {z}{k1}
# CHECK-NEXT: 1 1 0.50 vpunpckldq %ymm16, %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: 1 8 0.50 * vpunpckldq (%rax), %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: 1 8 0.50 * vpunpckldq (%rax){1to8}, %ymm17, %ymm19 {%k1} {z}
-# CHECK-NEXT: 1 1 1.00 * vscatterdps %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterdpd %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterqps %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterqpd %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterdps %ymm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterdpd %ymm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterqps %xmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterqpd %ymm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 27 9 8.25 * vscatterdps %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 17 6 4.00 * vscatterdpd %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 17 6 4.00 * vscatterqps %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 17 6 4.00 * vscatterqpd %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 48 14 12.25 * vscatterdps %ymm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 28 9 6.00 * vscatterdpd %ymm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 27 9 6.50 * vscatterqps %xmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 27 9 6.50 * vscatterqpd %ymm1, (%rdx,%ymm0,4) {%k1}
# CHECK-NEXT: 1 2 1.00 vshuff32x4 $0, %ymm16, %ymm17, %ymm19
# CHECK-NEXT: 3 9 1.00 * vshuff32x4 $0, (%rax), %ymm17, %ymm19
# CHECK-NEXT: 3 9 1.00 * vshuff32x4 $0, (%rax){1to8}, %ymm17, %ymm19
@@ -3614,7 +3614,7 @@ vunpcklps (%rax){1to8}, %ymm17, %ymm19 {z}{k1}
# CHECK: Resource pressure per iteration:
# CHECK-NEXT: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12.0] [12.1] [13] [14.0] [14.1] [14.2] [15.0] [15.1] [15.2] [16.0] [16.1]
-# CHECK-NEXT: 10.67 10.67 10.67 - - - - - 208.00 1084.00 637.50 261.50 509.50 509.50 32.00 355.67 355.67 355.67 334.33 334.33 334.33 32.00 32.00
+# CHECK-NEXT: 40.00 40.00 40.00 - - - - - 393.50 1269.50 823.00 447.00 569.50 569.50 92.00 379.67 379.67 379.67 349.00 349.00 349.00 46.00 46.00
# CHECK: Resource pressure by instruction:
# CHECK-NEXT: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12.0] [12.1] [13] [14.0] [14.1] [14.2] [15.0] [15.1] [15.2] [16.0] [16.1] Instructions:
@@ -4098,14 +4098,14 @@ vunpcklps (%rax){1to8}, %ymm17, %ymm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - 0.50 0.50 - - - - - - - - - - - - - vfmadd231ps %ymm16, %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - 0.50 0.50 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vfmadd231ps (%rax), %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - 0.50 0.50 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vfmadd231ps (%rax){1to8}, %ymm17, %ymm19 {%k1} {z}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdpd (%rax,%xmm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdps (%rax,%ymm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqpd (%rax,%ymm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqps (%rax,%ymm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdpd (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdps (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqpd (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqps (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vgatherdpd (%rax,%xmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 9.25 9.25 9.25 9.25 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vgatherdps (%rax,%ymm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vgatherqpd (%rax,%ymm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vgatherqps (%rax,%ymm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vgatherdpd (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vgatherdps (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vgatherqpd (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vgatherqps (%rax,%xmm1,2), %xmm2 {%k1}
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - - - - - - - - - - - - vmaxpd %xmm16, %xmm17, %xmm19
# CHECK-NEXT: - - - - - - - - 0.50 0.50 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vmaxpd (%rax), %xmm17, %xmm19
# CHECK-NEXT: - - - - - - - - 0.50 0.50 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vmaxpd (%rax){1to2}, %xmm17, %xmm19
@@ -4662,14 +4662,14 @@ vunpcklps (%rax){1to8}, %ymm17, %ymm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - - - - - - - - - - - - vpermq %ymm16, %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpermq (%rax), %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpermq (%rax){1to4}, %ymm17, %ymm19 {%k1} {z}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdq (%rax,%xmm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdd (%rax,%ymm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqq (%rax,%ymm1,2), %ymm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqd (%rax,%ymm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdq (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdd (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqq (%rax,%xmm1,2), %xmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqd (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vpgatherdq (%rax,%xmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 9.25 9.25 9.25 9.25 4.00 4.00 - 2.67 2.67 2.67 2.67 2.67 2.67 - - vpgatherdd (%rax,%ymm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vpgatherqq (%rax,%ymm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vpgatherqd (%rax,%ymm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vpgatherdq (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 5.00 5.00 5.00 5.00 2.00 2.00 - 1.33 1.33 1.33 1.33 1.33 1.33 - - vpgatherdd (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vpgatherqq (%rax,%xmm1,2), %xmm2 {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 - 0.67 0.67 0.67 0.67 0.67 0.67 - - vpgatherqd (%rax,%xmm1,2), %xmm2 {%k1}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - - - - - - - - - - - - vpmovdb %xmm19, %xmm16
# CHECK-NEXT: - - - - - - - - - 1.50 1.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpmovdb %xmm19, (%rax)
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - - - - - - - - - - - - vpmovdb %xmm19, %xmm16 {%k1}
@@ -4958,14 +4958,14 @@ vunpcklps (%rax){1to8}, %ymm17, %ymm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - 0.50 - - 0.50 - - - - - - - - - - - vpmulld %ymm16, %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - 0.50 - - 0.50 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpmulld (%rax), %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - 0.50 - - 0.50 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpmulld (%rax){1to8}, %ymm17, %ymm19 {%k1} {z}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterdd %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterdq %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterqd %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterqq %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterdd %ymm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterdq %ymm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterqd %xmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterqq %ymm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 8.25 8.25 8.25 8.25 2.00 2.00 4.00 1.33 1.33 1.33 - - - 2.00 2.00 vpscatterdd %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 2.00 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterdq %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 2.00 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterqd %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 2.00 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterqq %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 12.25 12.25 12.25 12.25 4.00 4.00 8.00 2.67 2.67 2.67 - - - 4.00 4.00 vpscatterdd %ymm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 6.00 6.00 6.00 6.00 2.00 2.00 4.00 1.33 1.33 1.33 - - - 2.00 2.00 vpscatterdq %ymm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 6.50 6.50 6.50 6.50 2.00 2.00 4.00 1.33 1.33 1.33 - - - 2.00 2.00 vpscatterqd %xmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 6.50 6.50 6.50 6.50 2.00 2.00 4.00 1.33 1.33 1.33 - - - 2.00 2.00 vpscatterqq %ymm1, (%rdx,%ymm0,4) {%k1}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - - - - - - - - - - - - vpshufd $0, %xmm16, %xmm19
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpshufd $0, (%rax), %xmm19
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpshufd $0, (%rax){1to4}, %xmm19
@@ -5104,14 +5104,14 @@ vunpcklps (%rax){1to8}, %ymm17, %ymm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - - - - - - - - - - - - vpunpckldq %ymm16, %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpunpckldq (%rax), %ymm17, %ymm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpunpckldq (%rax){1to8}, %ymm17, %ymm19 {%k1} {z}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterdps %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterdpd %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterqps %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterqpd %xmm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterdps %ymm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterdpd %ymm1, (%rdx,%xmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterqps %xmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vscatterqpd %ymm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 8.25 8.25 8.25 8.25 2.00 2.00 4.00 1.33 1.33 1.33 - - - 2.00 2.00 vscatterdps %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 2.00 0.67 0.67 0.67 - - - 1.00 1.00 vscatterdpd %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 2.00 0.67 0.67 0.67 - - - 1.00 1.00 vscatterqps %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 0.67 0.67 0.67 - - - - - 4.00 4.00 4.00 4.00 1.00 1.00 2.00 0.67 0.67 0.67 - - - 1.00 1.00 vscatterqpd %xmm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 2.67 2.67 2.67 - - - - - 12.25 12.25 12.25 12.25 4.00 4.00 8.00 2.67 2.67 2.67 - - - 4.00 4.00 vscatterdps %ymm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 6.00 6.00 6.00 6.00 2.00 2.00 4.00 1.33 1.33 1.33 - - - 2.00 2.00 vscatterdpd %ymm1, (%rdx,%xmm0,4) {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 6.50 6.50 6.50 6.50 2.00 2.00 4.00 1.33 1.33 1.33 - - - 2.00 2.00 vscatterqps %xmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 1.33 1.33 1.33 - - - - - 6.50 6.50 6.50 6.50 2.00 2.00 4.00 1.33 1.33 1.33 - - - 2.00 2.00 vscatterqpd %ymm1, (%rdx,%ymm0,4) {%k1}
# CHECK-NEXT: - - - - - - - - - 1.00 - - - - - - - - - - - - - vshuff32x4 $0, %ymm16, %ymm17, %ymm19
# CHECK-NEXT: - - - - - - - - - 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vshuff32x4 $0, (%rax), %ymm17, %ymm19
# CHECK-NEXT: - - - - - - - - - 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vshuff32x4 $0, (%rax){1to8}, %ymm17, %ymm19
>From a8c18029e855c7fbfc47af4950a2411c2d9dd63a Mon Sep 17 00:00:00 2001
From: Sumukh Bharadwaj <Sumukh.Bharadwaj at amd.com>
Date: Wed, 2 Sep 2026 14:37:46 +0530
Subject: [PATCH 2/2] [X86][SchedModel] Test Znver4 gather mask writeback
The per-shape InstRW overrides must retain the gather's second scheduling
write. Add a dependent llvm-mca timeline check so dropping it cannot
silently replace the five-cycle mask dependency with the data-result
latency.
---
.../llvm-mca/X86/Znver4/gather-mask-writeback.s | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
create mode 100644 llvm/test/tools/llvm-mca/X86/Znver4/gather-mask-writeback.s
diff --git a/llvm/test/tools/llvm-mca/X86/Znver4/gather-mask-writeback.s b/llvm/test/tools/llvm-mca/X86/Znver4/gather-mask-writeback.s
new file mode 100644
index 0000000000000..327332194c16d
--- /dev/null
+++ b/llvm/test/tools/llvm-mca/X86/Znver4/gather-mask-writeback.s
@@ -0,0 +1,16 @@
+# RUN: llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=znver4 \
+# RUN: -timeline -iterations=1 < %s | FileCheck %s
+
+# The 42 micro-ops delay dispatching the dependent vpaddd until after the
+# five-cycle mask writeback is ready. It should therefore execute as soon as it
+# is dispatched. If the gather's second scheduling write is dropped, vpaddd
+# instead waits for the 21-cycle data result.
+
+vpgatherdd %ymm0, (%rax,%ymm1,4), %ymm2
+vpaddd %ymm0, %ymm3, %ymm4
+
+# CHECK: Instructions:
+# CHECK: 42 21 8.25 * vpgatherdd
+# CHECK: Timeline view:
+# CHECK: [0,0] {{.*}} vpgatherdd
+# CHECK-NEXT: [0,1] {{.*}}DeE{{-*}}R vpaddd
More information about the llvm-commits
mailing list