[llvm] [X86][SchedModel] Add Znver4/Znver5 masked gather/scatter overrides (PR #212997)
via llvm-commits
llvm-commits at lists.llvm.org
Thu Jul 30 04:04:14 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-backend-x86
Author: Sumukh J Bharadwaj (amd-subharad)
<details>
<summary>Changes</summary>
## Summary
The Znver4 scheduling model (also used by Znver5) had no overrides for the
AVX-512 masked gather/scatter instructions, so they fell back to the default
vector load/store write. This adds per-shape `SchedWriteRes` overrides for the
masked gather (`VPGATHER*` / `VGATHER*`) and scatter (`VPSCATTER*` /
`VSCATTER*`) instructions covering every EVEX shape.
## Motivation
On Zen4/Zen5 these are microcoded sequences whose throughput, uops and latency
scale strongly with the number of elements and the element width, so the
default cost is far from the real hardware. Giving the scheduler and llvm-mca
honest numbers is a prerequisite for driving the TTI gather/scatter cost model
from the schedule model (follow-up PR).
## Implementation notes
- Override names follow the existing mnemonic + EVEX-width convention already
used in `X86ScheduleZnver4.td` (e.g. `Zn4WriteVPGATHERDDZ128`), with the FP
variant sharing the same `InstRW` as the integer representative.
- Entries are keyed by `(#elements, element width)` and shared between the
dword- and qword-index encodings of a shape, since the microcoded cost
tracks element count and width rather than index width.
- Throughput and uop counts were measured on Znver5 (Ryzen 9 9950X, which
reuses `Znver4Model`) with mask-reloading microbenchmarks under perf;
reciprocal throughput is reproduced through load-/store-pipe and AGU
occupancy via `ReleaseAtCycles`. Scatter latency is estimated (throughput
and uops are measured).
- The 2-element (128-bit) shapes are force-scalarised by the vectoriser and
were not measured; their values are extrapolated as half the 4-element entry,
purely so llvm-mca stays self-consistent.
## Tests
- `llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s`
- `llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512vl.s`
regenerated to reflect the new gather/scatter numbers.
---
Patch is 53.43 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/212997.diff
3 Files Affected:
- (modified) llvm/lib/Target/X86/X86ScheduleZnver4.td (+105)
- (modified) llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s (+33-33)
- (modified) llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512vl.s (+65-65)
``````````diff
diff --git a/llvm/lib/Target/X86/X86ScheduleZnver4.td b/llvm/lib/Target/X86/X86ScheduleZnver4.td
index ac4d31de8dbfe..1d058397f3455 100644
--- a/llvm/lib/Target/X86/X86ScheduleZnver4.td
+++ b/llvm/lib/Target/X86/X86ScheduleZnver4.td
@@ -509,6 +509,111 @@ defm : Zn4WriteResInt<WriteLoad, [Zn4AGU012, Zn4Load], !add(Znver4Model.LoadLate
// Does not cost anything by itself, only has latency, matching that of the WriteLoad,
defm : Zn4WriteResInt<WriteVecMaskedGatherWriteback, [], !add(Znver4Model.LoadLatency, 1), [], 0>;
+// AVX-512 masked GATHER / SCATTER, per shape.
+//
+// Zen4/Zen5 implement these as microcoded sequences (~4-5 macro-ops per
+// element), so the cost is a function of the number of elements and the
+// element width, not of the index width. Throughput and uops were measured on
+// Znver5 (Ryzen 9 9950X, which reuses Znver4Model) with mask-reloading
+// microbenchmarks under perf:
+// shape gather: tput(cyc) uops lat scatter: tput(cyc) uops
+// v4x32 4.0 20 17 6.0 28
+// v8x32 6.2 33 23 9.0 49
+// v16x32 12.7 65 30 17.0 89
+// v4x64 4.0 20 19 5.0 28
+// v8x64 8.0 41 25 9.0 49
+// Reciprocal throughput is reproduced through load-pipe (Zn4Load, 3 units) /
+// store-pipe (Zn4Store, 2 units) and AGU (Zn4AGU012, 3 units) occupancy:
+// ReleaseAtCycles = round(tput * NumUnits). Scatter latency is estimated
+// (throughput and uops are measured).
+//
+// Each entry is keyed by (#elements, element width); the dword- and
+// qword-index encodings of a shape share it (e.g. VPGATHERDQ and VPGATHERQQ
+// for 8 x i64). The 2-element (128-bit) shapes are force-scalarised by the
+// vectoriser and were not measured; their values are extrapolated as half the
+// 4-element entry and exist only so llvm-mca stays self-consistent.
+def Zn4WriteVPGATHERQDZ128 : SchedWriteRes<[Zn4AGU012, Zn4Load]> {
+ let ReleaseAtCycles = [6, 6]; let Latency = 14; let NumMicroOps = 10;
+}
+def : InstRW<[Zn4WriteVPGATHERQDZ128, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQDZ128rm, VGATHERQPSZ128rm)>;
+def Zn4WriteVPGATHERDDZ128 : SchedWriteRes<[Zn4AGU012, Zn4Load]> {
+ let ReleaseAtCycles = [12, 12]; let Latency = 17; let NumMicroOps = 20;
+}
+def : InstRW<[Zn4WriteVPGATHERDDZ128, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERDDZ128rm, VGATHERDPSZ128rm,
+ VPGATHERQDZ256rm, VGATHERQPSZ256rm)>;
+def Zn4WriteVPGATHERDDZ256 : SchedWriteRes<[Zn4AGU012, Zn4Load]> {
+ let ReleaseAtCycles = [19, 19]; let Latency = 23; let NumMicroOps = 33;
+}
+def : InstRW<[Zn4WriteVPGATHERDDZ256, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERDDZ256rm, VGATHERDPSZ256rm,
+ VPGATHERQDZrm, VGATHERQPSZrm)>;
+def Zn4WriteVPGATHERDDZ : SchedWriteRes<[Zn4AGU012, Zn4Load]> {
+ let ReleaseAtCycles = [38, 38]; let Latency = 30; let NumMicroOps = 65;
+}
+def : InstRW<[Zn4WriteVPGATHERDDZ, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERDDZrm, VGATHERDPSZrm)>;
+def Zn4WriteVPGATHERQQZ128 : SchedWriteRes<[Zn4AGU012, Zn4Load]> {
+ let ReleaseAtCycles = [6, 6]; let Latency = 15; let NumMicroOps = 10;
+}
+def : InstRW<[Zn4WriteVPGATHERQQZ128, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQQZ128rm, VGATHERQPDZ128rm,
+ VPGATHERDQZ128rm, VGATHERDPDZ128rm)>;
+def Zn4WriteVPGATHERQQZ256 : SchedWriteRes<[Zn4AGU012, Zn4Load]> {
+ let ReleaseAtCycles = [12, 12]; let Latency = 19; let NumMicroOps = 20;
+}
+def : InstRW<[Zn4WriteVPGATHERQQZ256, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQQZ256rm, VGATHERQPDZ256rm,
+ VPGATHERDQZ256rm, VGATHERDPDZ256rm)>;
+def Zn4WriteVPGATHERQQZ : SchedWriteRes<[Zn4AGU012, Zn4Load]> {
+ let ReleaseAtCycles = [24, 24]; let Latency = 25; let NumMicroOps = 41;
+}
+def : InstRW<[Zn4WriteVPGATHERQQZ, WriteVecMaskedGatherWriteback],
+ (instrs VPGATHERQQZrm, VGATHERQPDZrm,
+ VPGATHERDQZrm, VGATHERDPDZrm)>;
+
+def Zn4WriteVPSCATTERQDZ128 : SchedWriteRes<[Zn4AGU012, Zn4Store]> {
+ let ReleaseAtCycles = [9, 6]; let Latency = 6; let NumMicroOps = 14;
+}
+def : InstRW<[Zn4WriteVPSCATTERQDZ128],
+ (instrs VPSCATTERQDZ128mr, VSCATTERQPSZ128mr)>;
+def Zn4WriteVPSCATTERDDZ128 : SchedWriteRes<[Zn4AGU012, Zn4Store]> {
+ let ReleaseAtCycles = [18, 12]; let Latency = 10; let NumMicroOps = 28;
+}
+def : InstRW<[Zn4WriteVPSCATTERDDZ128],
+ (instrs VPSCATTERDDZ128mr, VSCATTERDPSZ128mr,
+ VPSCATTERQDZ256mr, VSCATTERQPSZ256mr)>;
+def Zn4WriteVPSCATTERDDZ256 : SchedWriteRes<[Zn4AGU012, Zn4Store]> {
+ let ReleaseAtCycles = [27, 18]; let Latency = 14; let NumMicroOps = 49;
+}
+def : InstRW<[Zn4WriteVPSCATTERDDZ256],
+ (instrs VPSCATTERDDZ256mr, VSCATTERDPSZ256mr,
+ VPSCATTERQDZmr, VSCATTERQPSZmr)>;
+def Zn4WriteVPSCATTERDDZ : SchedWriteRes<[Zn4AGU012, Zn4Store]> {
+ let ReleaseAtCycles = [51, 34]; let Latency = 24; let NumMicroOps = 89;
+}
+def : InstRW<[Zn4WriteVPSCATTERDDZ],
+ (instrs VPSCATTERDDZmr, VSCATTERDPSZmr)>;
+def Zn4WriteVPSCATTERQQZ128 : SchedWriteRes<[Zn4AGU012, Zn4Store]> {
+ let ReleaseAtCycles = [8, 5]; let Latency = 5; let NumMicroOps = 14;
+}
+def : InstRW<[Zn4WriteVPSCATTERQQZ128],
+ (instrs VPSCATTERQQZ128mr, VSCATTERQPDZ128mr,
+ VPSCATTERDQZ128mr, VSCATTERDPDZ128mr)>;
+def Zn4WriteVPSCATTERQQZ256 : SchedWriteRes<[Zn4AGU012, Zn4Store]> {
+ let ReleaseAtCycles = [15, 10]; let Latency = 9; let NumMicroOps = 28;
+}
+def : InstRW<[Zn4WriteVPSCATTERQQZ256],
+ (instrs VPSCATTERQQZ256mr, VSCATTERQPDZ256mr,
+ VPSCATTERDQZ256mr, VSCATTERDPDZ256mr)>;
+def Zn4WriteVPSCATTERQQZ : SchedWriteRes<[Zn4AGU012, Zn4Store]> {
+ let ReleaseAtCycles = [27, 18]; let Latency = 14; let NumMicroOps = 49;
+}
+def : InstRW<[Zn4WriteVPSCATTERQQZ],
+ (instrs VPSCATTERQQZmr, VSCATTERQPDZmr,
+ VPSCATTERDQZmr, VSCATTERDPDZmr)>;
+
def Zn4WriteMOVSlow : SchedWriteRes<[Zn4AGU012, Zn4Load]> {
let Latency = !add(Znver4Model.LoadLatency, 1);
let ReleaseAtCycles = [3, 1];
diff --git a/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s b/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s
index 14b8e5f36c666..22983b0161a2b 100644
--- a/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s
+++ b/llvm/test/tools/llvm-mca/X86/Znver4/resources-avx512.s
@@ -1495,10 +1495,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: 1 4 1.00 vfmadd231ps %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 1 11 1.00 * vfmadd231ps (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 1 11 1.00 * vfmadd231ps (%rax){1to16}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 1 5 0.33 * vgatherdpd (%rax,%ymm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherdps (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherqpd (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vgatherqps (%rax,%zmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 41 25 8.00 * vgatherdpd (%rax,%ymm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 65 30 12.67 * vgatherdps (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 41 25 8.00 * vgatherqpd (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 33 23 6.33 * vgatherqps (%rax,%zmm1,2), %ymm2 {%k1}
# CHECK-NEXT: 1 2 1.00 vmaxpd %zmm16, %zmm17, %zmm19
# CHECK-NEXT: 1 9 1.00 * vmaxpd (%rax), %zmm17, %zmm19
# CHECK-NEXT: 1 9 1.00 * vmaxpd (%rax){1to8}, %zmm17, %zmm19
@@ -1732,10 +1732,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: 1 1 0.50 vpcmpequq %zmm0, %zmm1, %k2 {%k3}
# CHECK-NEXT: 1 8 0.50 * vpcmpequq (%rax), %zmm1, %k2 {%k3}
# CHECK-NEXT: 1 8 0.50 * vpcmpequq (%rax){1to8}, %zmm1, %k2 {%k3}
-# CHECK-NEXT: 1 5 0.33 * vpgatherdq (%rax,%ymm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherdd (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherqq (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 1 5 0.33 * vpgatherqd (%rax,%zmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 41 25 8.00 * vpgatherdq (%rax,%ymm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 65 30 12.67 * vpgatherdd (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 41 25 8.00 * vpgatherqq (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 33 23 6.33 * vpgatherqd (%rax,%zmm1,2), %ymm2 {%k1}
# CHECK-NEXT: 1 5 1.00 vpmovdb %zmm19, %xmm16
# CHECK-NEXT: 1 11 1.50 * vpmovdb %zmm19, (%rax)
# CHECK-NEXT: 1 5 1.00 vpmovdb %zmm19, %xmm16 {%k1}
@@ -1970,10 +1970,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: 2 1 0.50 vpermq %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 2 8 0.50 * vpermq (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 2 8 0.50 * vpermq (%rax){1to8}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 1 1 1.00 * vpscatterdd %zmm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterdq %zmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterqd %ymm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vpscatterqq %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 89 24 17.00 * vpscatterdd %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 49 14 9.00 * vpscatterdq %zmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 49 14 9.00 * vpscatterqd %ymm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 49 14 9.00 * vpscatterqq %zmm1, (%rdx,%zmm0,4) {%k1}
# CHECK-NEXT: 1 1 1.00 vpshufd $0, %zmm16, %zmm19
# CHECK-NEXT: 1 8 1.00 * vpshufd $0, (%rax), %zmm19
# CHECK-NEXT: 1 8 1.00 * vpshufd $0, (%rax){1to16}, %zmm19
@@ -2037,10 +2037,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: 1 1 1.00 vpunpcklqdq %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 1 8 1.00 * vpunpcklqdq (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: 1 8 1.00 * vpunpcklqdq (%rax){1to8}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 1 1 1.00 * vscatterdps %zmm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterdpd %zmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterqps %ymm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 1 1 1.00 * vscatterqpd %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 89 24 17.00 * vscatterdps %zmm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 49 14 9.00 * vscatterdpd %zmm1, (%rdx,%ymm0,4) {%k1}
+# CHECK-NEXT: 49 14 9.00 * vscatterqps %ymm1, (%rdx,%zmm0,4) {%k1}
+# CHECK-NEXT: 49 14 9.00 * vscatterqpd %zmm1, (%rdx,%zmm0,4) {%k1}
# CHECK-NEXT: 1 2 1.00 vshuff32x4 $0, %zmm16, %zmm17, %zmm19
# CHECK-NEXT: 3 9 1.00 * vshuff32x4 $0, (%rax), %zmm17, %zmm19
# CHECK-NEXT: 3 9 1.00 * vshuff32x4 $0, (%rax){1to16}, %zmm17, %zmm19
@@ -2233,7 +2233,7 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK: Resource pressure per iteration:
# CHECK-NEXT: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12.0] [12.1] [13] [14.0] [14.1] [14.2] [15.0] [15.1] [15.2] [16.0] [16.1]
-# CHECK-NEXT: 5.33 5.33 5.33 - - - - - 219.50 1119.00 676.50 351.00 312.50 312.50 17.00 215.67 215.67 215.67 204.67 204.67 204.67 16.50 16.50
+# CHECK-NEXT: 158.00 158.00 158.00 - - - - - 219.50 1119.00 676.50 351.00 312.50 312.50 17.00 336.33 336.33 336.33 272.00 272.00 272.00 96.50 96.50
# CHECK: Resource pressure by instruction:
# CHECK-NEXT: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12.0] [12.1] [13] [14.0] [14.1] [14.2] [15.0] [15.1] [15.2] [16.0] [16.1] Instructions:
@@ -2552,10 +2552,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - - - - - - - - - - - - vfmadd231ps %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vfmadd231ps (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vfmadd231ps (%rax){1to16}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdpd (%rax,%ymm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherdps (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqpd (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vgatherqps (%rax,%zmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 8.00 8.00 8.00 - - - - - - - - - - - - 8.00 8.00 8.00 8.00 8.00 8.00 - - vgatherdpd (%rax,%ymm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 12.67 12.67 12.67 - - - - - - - - - - - - 12.67 12.67 12.67 12.67 12.67 12.67 - - vgatherdps (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 8.00 8.00 8.00 - - - - - - - - - - - - 8.00 8.00 8.00 8.00 8.00 8.00 - - vgatherqpd (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 6.33 6.33 6.33 - - - - - - - - - - - - 6.33 6.33 6.33 6.33 6.33 6.33 - - vgatherqps (%rax,%zmm1,2), %ymm2 {%k1}
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - - - - - - - - - - - - vmaxpd %zmm16, %zmm17, %zmm19
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vmaxpd (%rax), %zmm17, %zmm19
# CHECK-NEXT: - - - - - - - - 1.00 1.00 - - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vmaxpd (%rax){1to8}, %zmm17, %zmm19
@@ -2789,10 +2789,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - 0.50 0.50 0.50 0.50 - - - - - - - - - - - vpcmpequq %zmm0, %zmm1, %k2 {%k3}
# CHECK-NEXT: - - - - - - - - 0.50 0.50 0.50 0.50 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpcmpequq (%rax), %zmm1, %k2 {%k3}
# CHECK-NEXT: - - - - - - - - 0.50 0.50 0.50 0.50 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpcmpequq (%rax){1to8}, %zmm1, %k2 {%k3}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdq (%rax,%ymm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherdd (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqq (%rax,%zmm1,2), %zmm2 {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpgatherqd (%rax,%zmm1,2), %ymm2 {%k1}
+# CHECK-NEXT: 8.00 8.00 8.00 - - - - - - - - - - - - 8.00 8.00 8.00 8.00 8.00 8.00 - - vpgatherdq (%rax,%ymm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 12.67 12.67 12.67 - - - - - - - - - - - - 12.67 12.67 12.67 12.67 12.67 12.67 - - vpgatherdd (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 8.00 8.00 8.00 - - - - - - - - - - - - 8.00 8.00 8.00 8.00 8.00 8.00 - - vpgatherqq (%rax,%zmm1,2), %zmm2 {%k1}
+# CHECK-NEXT: 6.33 6.33 6.33 - - - - - - - - - - - - 6.33 6.33 6.33 6.33 6.33 6.33 - - vpgatherqd (%rax,%zmm1,2), %ymm2 {%k1}
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - - - - - - - - - - - - vpmovdb %zmm19, %xmm16
# CHECK-NEXT: - - - - - - - - - 1.50 1.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpmovdb %zmm19, (%rax)
# CHECK-NEXT: - - - - - - - - - 1.00 1.00 - - - - - - - - - - - - vpmovdb %zmm19, %xmm16 {%k1}
@@ -3027,10 +3027,10 @@ vunpcklps (%rax){1to16}, %zmm17, %zmm19 {z}{k1}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - - - - - - - - - - - - vpermq %zmm16, %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpermq (%rax), %zmm17, %zmm19 {%k1} {z}
# CHECK-NEXT: - - - - - - - - - 0.50 0.50 - 0.50 0.50 - 0.33 0.33 0.33 0.33 0.33 0.33 - - vpermq (%rax){1to8}, %zmm17, %zmm19 {%k1} {z}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterdd %zmm1, (%rdx,%zmm0,4) {%k1}
-# CHECK-NEXT: 0.33 0.33 0.33 - - - - - - - - - - - - 0.67 0.67 0.67 - - - 1.00 1.00 vpscatterdq %zmm1, (%rdx,%ymm0,4) {%k1}
-# CHECK-NE...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/212997
More information about the llvm-commits
mailing list