[llvm] [X86] Correct folded insert costs for partial vector loads (PR #220174)
Milin Bhade via llvm-commits
llvm-commits at lists.llvm.org
Tue Sep 8 22:52:53 PDT 2026
https://github.com/mbhade-amd updated https://github.com/llvm/llvm-project/pull/220174
>From 7144adce79f0c28bc0e28084c701efb5a23f99a0 Mon Sep 17 00:00:00 2001
From: mbhade <mbhade at amd.com>
Date: Tue, 25 Aug 2026 14:35:13 +0530
Subject: [PATCH 1/3] [X86] Correct folded insert costs for partial vector
loads
Recent SLP support for non-power-of-two nodes can form width-3 trees and relies on target costs to determine profitability:
https://github.com/llvm/llvm-project/pull/194189
X86 getMemoryOpCost counted a non-0th 1/2/4-byte PINSR*(mem) chunk as both a memory operation and a separate lane insertion. This overpricing caused SLP to reject the width-3 shared-weight RGB reduction.
Recognize non-0th folded load-insert chunks and charge each once. Leading sub-dword loads, stores, and 8-byte chunks remain unchanged.
---
.../lib/Target/X86/X86TargetTransformInfo.cpp | 21 +-
.../test/Analysis/CostModel/X86/load_store.ll | 1306 ++++++++---------
.../CostModel/X86/nonpow2-load-fold.ll | 68 +
.../interleaved-load-i16-stride-3.ll | 4 +-
.../interleaved-load-i16-stride-5.ll | 2 +-
.../interleaved-load-i16-stride-7.ll | 2 +-
.../CostModel/interleaved-load-i8-stride-3.ll | 8 +-
.../CostModel/interleaved-load-i8-stride-5.ll | 4 +-
.../CostModel/interleaved-load-i8-stride-6.ll | 4 +-
.../CostModel/interleaved-load-i8-stride-7.ll | 4 +-
.../X86/nonpow2-shared-weight-reduction.ll | 150 ++
11 files changed, 902 insertions(+), 671 deletions(-)
create mode 100644 llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
create mode 100644 llvm/test/Transforms/SLPVectorizer/X86/nonpow2-shared-weight-reduction.ll
diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index 8e0cf1fc5a153..36b01e3ef9859 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -5548,9 +5548,24 @@ InstructionCost X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
// Sandybridge.
// Sub-32-bit loads/stores will be slower either with PINSR*/PEXTR* or
// will be scalarized.
+ //
+ // For a vector load, each non-0th 1/2/4-byte in-lane remainder chunk is
+ // materialized by a *single* folded PINSR*(mem) that both loads and
+ // inserts the lane (1B->PINSRB, 2B->PINSRW, 4B->PINSRD; the byte and
+ // dword folds need SSE4.1, the word fold only SSE2).
+ // When that fold is available the chunk is one instruction, so it must be
+ // priced once here (as a plain load) and the separate lane-insert charge
+ // below must be skipped - otherwise the folded insert is double-counted.
+ // Stores (the symmetric PEXTR*(mem) fold) are left unchanged here.
+ bool Is0thSubVec = (NumEltDone() % LT.second.getVectorNumElements()) == 0;
+ bool FoldedInLaneInsert =
+ IsLoad && !Is0thSubVec &&
+ ((CurrOpSizeBytes == 1 && ST->hasSSE41()) ||
+ (CurrOpSizeBytes == 2 && ST->hasSSE2()) ||
+ (CurrOpSizeBytes == 4 && ST->hasSSE41()));
if (CurrOpSizeBytes == 32 && ST->isUnalignedMem32Slow())
Cost += 2;
- else if (CurrOpSizeBytes < 4)
+ else if (CurrOpSizeBytes < 4 && !FoldedInLaneInsert)
Cost += 2;
else
Cost += 1;
@@ -5560,8 +5575,6 @@ InstructionCost X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
if (IsLoad && OpInfo.isUniform())
return Cost;
- bool Is0thSubVec = (NumEltDone() % LT.second.getVectorNumElements()) == 0;
-
// If we have fully processed the previous reg, we need to replenish it.
if (SubVecEltsLeft == 0) {
SubVecEltsLeft += CurrVecTy->getNumElements();
@@ -5577,7 +5590,7 @@ InstructionCost X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
// for smaller widths (32/16/8) we have to insert/extract them separately.
// Again, it's free for the 0'th subreg (if op is 32/64 bit wide,
// but let's pretend that it is also true for 16/8 bit wide ops...)
- if (CurrOpSizeBytes <= 32 / 8 && !Is0thSubVec) {
+ if (CurrOpSizeBytes <= 32 / 8 && !Is0thSubVec && !FoldedInLaneInsert) {
int NumEltDoneInCurrXMM = NumEltDone() % NumEltPerXMM;
assert(NumEltDoneInCurrXMM % CurrNumEltPerOp == 0 && "");
int CoalescedVecEltIdx = NumEltDoneInCurrXMM / CurrNumEltPerOp;
diff --git a/llvm/test/Analysis/CostModel/X86/load_store.ll b/llvm/test/Analysis/CostModel/X86/load_store.ll
index ef36bd7070724..0db96a06c578f 100644
--- a/llvm/test/Analysis/CostModel/X86/load_store.ll
+++ b/llvm/test/Analysis/CostModel/X86/load_store.ll
@@ -4222,120 +4222,120 @@ define i32 @loads_partial_align4(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %10 = load <7 x double>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %14 = load <5 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %15 = load <6 x i32>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %16 = load <7 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %17 = load <9 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %18 = load <10 x i32>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %19 = load <11 x i32>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %20 = load <12 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %21 = load <13 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %22 = load <14 x i32>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %23 = load <15 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %27 = load <5 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %28 = load <6 x float>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %29 = load <7 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %30 = load <9 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %31 = load <10 x float>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %32 = load <11 x float>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %33 = load <12 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %34 = load <13 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %35 = load <14 x float>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %36 = load <15 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %39 = load <3 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %41 = load <5 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %43 = load <7 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %44 = load <9 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %45 = load <10 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %46 = load <11 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %47 = load <12 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %50 = load <15 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %48 = load <13 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %49 = load <14 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %50 = load <15 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %51 = load <17 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %53 = load <19 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %55 = load <21 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %56 = load <22 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %57 = load <23 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %57 = load <23 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %59 = load <25 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %61 = load <27 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %63 = load <29 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %65 = load <31 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %64 = load <30 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %65 = load <31 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %67 = load <2 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %68 = load <3 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %70 = load <5 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %71 = load <6 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %72 = load <7 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %74 = load <9 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %75 = load <10 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %78 = load <13 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %79 = load <14 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %80 = load <15 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %76 = load <11 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %80 = load <15 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %81 = load <17 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %82 = load <18 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %83 = load <19 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %84 = load <20 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %87 = load <23 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %85 = load <21 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %86 = load <22 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %87 = load <23 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %88 = load <24 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %91 = load <27 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %93 = load <29 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %94 = load <30 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %95 = load <31 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %89 = load <25 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %90 = load <26 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %91 = load <27 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %92 = load <28 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %93 = load <29 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %94 = load <30 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %95 = load <31 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %96 = load <33 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %97 = load <34 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %98 = load <35 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %100 = load <37 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %101 = load <38 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %102 = load <39 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %102 = load <39 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %104 = load <41 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %105 = load <42 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %106 = load <43 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %107 = load <44 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %108 = load <45 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %109 = load <46 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %110 = load <47 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %106 = load <43 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %108 = load <45 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %109 = load <46 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %110 = load <47 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %112 = load <49 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %113 = load <50 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %114 = load <51 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %116 = load <53 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %117 = load <54 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %118 = load <55 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %116 = load <53 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %118 = load <55 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %120 = load <57 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %121 = load <58 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %124 = load <61 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %125 = load <62 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %126 = load <63 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %120 = load <57 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %122 = load <59 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %123 = load <60 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %124 = load <61 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %125 = load <62 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %126 = load <63 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %127 = load <3 x i1>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %128 = load <5 x i1>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %129 = load <6 x i1>, ptr undef, align 4
@@ -4408,120 +4408,120 @@ define i32 @loads_partial_align4(i32 %arg) {
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %10 = load <7 x double>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %14 = load <5 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %14 = load <5 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %15 = load <6 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %16 = load <7 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %17 = load <9 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %18 = load <10 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %19 = load <11 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %20 = load <12 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %21 = load <13 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %21 = load <13 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %22 = load <14 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %23 = load <15 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %27 = load <5 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %28 = load <6 x float>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %29 = load <7 x float>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %30 = load <9 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %31 = load <10 x float>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %32 = load <11 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %33 = load <12 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %34 = load <13 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %35 = load <14 x float>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %36 = load <15 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %39 = load <3 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %41 = load <5 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %43 = load <7 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %44 = load <9 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %45 = load <10 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %46 = load <11 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %45 = load <10 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %47 = load <12 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %48 = load <13 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %49 = load <14 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %50 = load <15 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %50 = load <15 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %51 = load <17 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %52 = load <18 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %53 = load <19 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %54 = load <20 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %57 = load <23 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %55 = load <21 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %56 = load <22 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %57 = load <23 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %58 = load <24 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %60 = load <26 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %61 = load <27 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %59 = load <25 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %63 = load <29 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %65 = load <31 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %64 = load <30 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %65 = load <31 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %67 = load <2 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %68 = load <3 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %70 = load <5 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %71 = load <6 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %72 = load <7 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %74 = load <9 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %75 = load <10 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %78 = load <13 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %79 = load <14 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %80 = load <15 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %81 = load <17 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %82 = load <18 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %84 = load <20 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %85 = load <21 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %86 = load <22 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %87 = load <23 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %76 = load <11 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %80 = load <15 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %83 = load <19 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %84 = load <20 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %87 = load <23 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %88 = load <24 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %89 = load <25 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %90 = load <26 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %91 = load <27 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %92 = load <28 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %93 = load <29 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %94 = load <30 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %95 = load <31 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %91 = load <27 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %93 = load <29 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %94 = load <30 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %95 = load <31 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %96 = load <33 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %97 = load <34 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %98 = load <35 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %99 = load <36 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %102 = load <39 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %100 = load <37 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %101 = load <38 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %102 = load <39 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %103 = load <40 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %106 = load <43 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %108 = load <45 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %109 = load <46 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %110 = load <47 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %104 = load <41 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %105 = load <42 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %106 = load <43 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %107 = load <44 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %108 = load <45 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %109 = load <46 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %110 = load <47 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %111 = load <48 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %114 = load <51 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %115 = load <52 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %116 = load <53 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %117 = load <54 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %118 = load <55 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %112 = load <49 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %113 = load <50 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %114 = load <51 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %116 = load <53 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %118 = load <55 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %120 = load <57 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %121 = load <58 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %124 = load <61 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %125 = load <62 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %126 = load <63 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %120 = load <57 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %122 = load <59 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %123 = load <60 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %124 = load <61 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %125 = load <62 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %126 = load <63 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %127 = load <3 x i1>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %128 = load <5 x i1>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %129 = load <6 x i1>, ptr undef, align 4
@@ -4594,120 +4594,120 @@ define i32 @loads_partial_align4(i32 %arg) {
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %10 = load <7 x double>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %14 = load <5 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %14 = load <5 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %15 = load <6 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %16 = load <7 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %17 = load <9 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %17 = load <9 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %18 = load <10 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %19 = load <11 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %20 = load <12 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %21 = load <13 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %21 = load <13 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %22 = load <14 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %23 = load <15 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %27 = load <5 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %28 = load <6 x float>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %29 = load <7 x float>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %30 = load <9 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %31 = load <10 x float>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %32 = load <11 x float>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %33 = load <12 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %34 = load <13 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %35 = load <14 x float>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %36 = load <15 x float>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %39 = load <3 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %41 = load <5 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %43 = load <7 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %44 = load <9 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %45 = load <10 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %46 = load <11 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %45 = load <10 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %47 = load <12 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %48 = load <13 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %49 = load <14 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %50 = load <15 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %51 = load <17 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %52 = load <18 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %53 = load <19 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %50 = load <15 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %51 = load <17 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %55 = load <21 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %56 = load <22 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %57 = load <23 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %57 = load <23 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %59 = load <25 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %60 = load <26 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %61 = load <27 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %60 = load <26 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %61 = load <27 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %62 = load <28 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %63 = load <29 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %64 = load <30 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %65 = load <31 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %63 = load <29 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %65 = load <31 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %67 = load <2 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %68 = load <3 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %70 = load <5 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %71 = load <6 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %72 = load <7 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %74 = load <9 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %75 = load <10 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %78 = load <13 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %79 = load <14 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %80 = load <15 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %81 = load <17 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %82 = load <18 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %84 = load <20 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %85 = load <21 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %86 = load <22 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %87 = load <23 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %76 = load <11 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %80 = load <15 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %83 = load <19 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %84 = load <20 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %87 = load <23 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %88 = load <24 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %89 = load <25 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %90 = load <26 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %91 = load <27 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %92 = load <28 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %93 = load <29 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %94 = load <30 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %95 = load <31 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %96 = load <33 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %97 = load <34 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %98 = load <35 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %99 = load <36 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %100 = load <37 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %101 = load <38 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %102 = load <39 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %91 = load <27 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %93 = load <29 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %94 = load <30 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %95 = load <31 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %96 = load <33 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %97 = load <34 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %98 = load <35 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %102 = load <39 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %104 = load <41 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %105 = load <42 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %106 = load <43 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %107 = load <44 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %108 = load <45 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %109 = load <46 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %110 = load <47 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %106 = load <43 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %108 = load <45 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %109 = load <46 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %110 = load <47 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %112 = load <49 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %113 = load <50 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %114 = load <51 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %115 = load <52 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %116 = load <53 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %117 = load <54 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %118 = load <55 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %114 = load <51 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %115 = load <52 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %116 = load <53 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %117 = load <54 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %118 = load <55 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %119 = load <56 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %120 = load <57 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %121 = load <58 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %122 = load <59 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %123 = load <60 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %124 = load <61 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %125 = load <62 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %126 = load <63 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %120 = load <57 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %121 = load <58 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %124 = load <61 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %125 = load <62 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %126 = load <63 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 4
@@ -4734,37 +4734,37 @@ define i32 @loads_partial_align4(i32 %arg) {
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %150 = load <29 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %151 = load <30 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %152 = load <31 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %153 = load <33 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %154 = load <34 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %155 = load <35 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %156 = load <36 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %157 = load <37 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %158 = load <38 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %159 = load <39 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %160 = load <40 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %161 = load <41 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %162 = load <42 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %163 = load <43 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %164 = load <44 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %165 = load <45 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %166 = load <46 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %167 = load <47 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %168 = load <48 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %169 = load <49 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %170 = load <50 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %171 = load <51 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %172 = load <52 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %173 = load <53 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %174 = load <54 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %175 = load <55 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %176 = load <56 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %177 = load <57 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %178 = load <58 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %179 = load <59 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %180 = load <60 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %181 = load <61 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %182 = load <62 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %183 = load <63 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %153 = load <33 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %154 = load <34 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %155 = load <35 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %156 = load <36 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %157 = load <37 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %158 = load <38 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %159 = load <39 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %160 = load <40 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %161 = load <41 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %162 = load <42 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %163 = load <43 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %164 = load <44 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %165 = load <45 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %166 = load <46 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %167 = load <47 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %168 = load <48 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %169 = load <49 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %170 = load <50 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %171 = load <51 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %172 = load <52 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %173 = load <53 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %174 = load <54 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %175 = load <55 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %176 = load <56 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %177 = load <57 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %178 = load <58 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %179 = load <59 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %180 = load <60 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %181 = load <61 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %182 = load <62 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %183 = load <63 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
load <1 x i64>, ptr undef, align 4
@@ -5308,94 +5308,94 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %36 = load <15 x float>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %39 = load <3 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %41 = load <5 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %42 = load <6 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %43 = load <7 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %43 = load <7 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %45 = load <10 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %46 = load <11 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %46 = load <11 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %47 = load <12 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %48 = load <13 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %48 = load <13 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %49 = load <14 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %50 = load <15 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %50 = load <15 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %51 = load <17 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %53 = load <19 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %55 = load <21 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %56 = load <22 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %57 = load <23 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %57 = load <23 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %61 = load <27 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %63 = load <29 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %64 = load <30 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %65 = load <31 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %65 = load <31 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %67 = load <2 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %68 = load <3 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 19 for instruction: %70 = load <5 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %71 = load <6 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %72 = load <7 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %72 = load <7 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 19 for instruction: %74 = load <9 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %75 = load <10 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %76 = load <11 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %76 = load <11 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %77 = load <12 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 24 for instruction: %78 = load <13 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %79 = load <14 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 27 for instruction: %80 = load <15 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %79 = load <14 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 25 for instruction: %80 = load <15 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %83 = load <19 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %84 = load <20 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %85 = load <21 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %86 = load <22 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %87 = load <23 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %86 = load <22 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %87 = load <23 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %88 = load <24 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %89 = load <25 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %90 = load <26 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %91 = load <27 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %90 = load <26 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %91 = load <27 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %92 = load <28 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 25 for instruction: %93 = load <29 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %94 = load <30 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 28 for instruction: %95 = load <31 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %94 = load <30 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 26 for instruction: %95 = load <31 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %96 = load <33 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %97 = load <34 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %98 = load <35 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %100 = load <37 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %101 = load <38 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 24 for instruction: %102 = load <39 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %102 = load <39 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %104 = load <41 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %105 = load <42 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 24 for instruction: %106 = load <43 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %106 = load <43 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %107 = load <44 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 26 for instruction: %108 = load <45 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %109 = load <46 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 29 for instruction: %110 = load <47 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %109 = load <46 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 27 for instruction: %110 = load <47 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %114 = load <51 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %116 = load <53 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %117 = load <54 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 25 for instruction: %118 = load <55 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %118 = load <55 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %120 = load <57 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %121 = load <58 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 25 for instruction: %122 = load <59 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %122 = load <59 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %123 = load <60 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 27 for instruction: %124 = load <61 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %125 = load <62 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 30 for instruction: %126 = load <63 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %125 = load <62 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 28 for instruction: %126 = load <63 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 1
@@ -5468,131 +5468,131 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %10 = load <7 x double>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %14 = load <5 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %15 = load <6 x i32>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %16 = load <7 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %17 = load <9 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %18 = load <10 x i32>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %19 = load <11 x i32>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %20 = load <12 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %21 = load <13 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %22 = load <14 x i32>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %23 = load <15 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %27 = load <5 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %28 = load <6 x float>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %29 = load <7 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %30 = load <9 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %31 = load <10 x float>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %32 = load <11 x float>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %33 = load <12 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %34 = load <13 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %35 = load <14 x float>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %36 = load <15 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %39 = load <3 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %41 = load <5 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %43 = load <7 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %45 = load <10 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %46 = load <11 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %46 = load <11 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %47 = load <12 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %48 = load <13 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %50 = load <15 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %48 = load <13 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %49 = load <14 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %50 = load <15 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %51 = load <17 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %53 = load <19 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %55 = load <21 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %56 = load <22 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %57 = load <23 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %57 = load <23 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %61 = load <27 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %63 = load <29 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %65 = load <31 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %64 = load <30 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %65 = load <31 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %67 = load <2 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %68 = load <3 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %68 = load <3 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %70 = load <5 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %71 = load <6 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %72 = load <7 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %74 = load <9 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %75 = load <10 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %76 = load <11 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %78 = load <13 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %79 = load <14 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %80 = load <15 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %80 = load <15 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %83 = load <19 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %84 = load <20 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %85 = load <21 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %86 = load <22 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %87 = load <23 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %85 = load <21 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %86 = load <22 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %87 = load <23 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %88 = load <24 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %89 = load <25 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %90 = load <26 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %91 = load <27 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %93 = load <29 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %94 = load <30 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %95 = load <31 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %89 = load <25 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %90 = load <26 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %91 = load <27 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %92 = load <28 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %93 = load <29 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %94 = load <30 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %95 = load <31 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %96 = load <33 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %97 = load <34 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %98 = load <35 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %98 = load <35 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %100 = load <37 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %101 = load <38 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %102 = load <39 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %102 = load <39 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %104 = load <41 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %105 = load <42 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %106 = load <43 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %107 = load <44 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %108 = load <45 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %109 = load <46 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %110 = load <47 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %106 = load <43 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %108 = load <45 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %109 = load <46 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %110 = load <47 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %114 = load <51 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %114 = load <51 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %116 = load <53 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %117 = load <54 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %118 = load <55 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %116 = load <53 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %118 = load <55 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %120 = load <57 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %121 = load <58 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %122 = load <59 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %124 = load <61 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %125 = load <62 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %126 = load <63 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %120 = load <57 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %123 = load <60 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %124 = load <61 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %125 = load <62 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %126 = load <63 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %130 = load <7 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %131 = load <9 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %132 = load <10 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %133 = load <11 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %134 = load <12 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %135 = load <13 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %136 = load <14 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %137 = load <15 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %131 = load <9 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %132 = load <10 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %133 = load <11 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %134 = load <12 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %135 = load <13 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %136 = load <14 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %137 = load <15 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %138 = load <17 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %139 = load <18 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %140 = load <19 x i1>, ptr undef, align 1
@@ -5601,13 +5601,13 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %143 = load <22 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %144 = load <23 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %145 = load <24 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %146 = load <25 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %147 = load <26 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %148 = load <27 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %149 = load <28 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %150 = load <29 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %151 = load <30 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %152 = load <31 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %146 = load <25 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %147 = load <26 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %148 = load <27 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %149 = load <28 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %150 = load <29 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %151 = load <30 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %152 = load <31 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %153 = load <33 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %154 = load <34 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %155 = load <35 x i1>, ptr undef, align 1
@@ -5616,13 +5616,13 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %158 = load <38 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %159 = load <39 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %160 = load <40 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %161 = load <41 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %162 = load <42 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %163 = load <43 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %164 = load <44 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %165 = load <45 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %166 = load <46 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %167 = load <47 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %161 = load <41 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %162 = load <42 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %163 = load <43 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %164 = load <44 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %165 = load <45 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %166 = load <46 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %167 = load <47 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %168 = load <48 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %169 = load <49 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %170 = load <50 x i1>, ptr undef, align 1
@@ -5632,13 +5632,13 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %174 = load <54 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %175 = load <55 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %176 = load <56 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %177 = load <57 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %178 = load <58 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %179 = load <59 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %180 = load <60 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %181 = load <61 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %182 = load <62 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %183 = load <63 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %177 = load <57 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %178 = load <58 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %179 = load <59 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %180 = load <60 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %181 = load <61 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %182 = load <62 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %183 = load <63 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
; AVX-LABEL: 'loads_partial_align1'
@@ -5654,146 +5654,146 @@ define i32 @loads_partial_align1(i32 %arg) {
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %10 = load <7 x double>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %14 = load <5 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %14 = load <5 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %15 = load <6 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %16 = load <7 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %17 = load <9 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %18 = load <10 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %19 = load <11 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %20 = load <12 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %21 = load <13 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %21 = load <13 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %22 = load <14 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %23 = load <15 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %27 = load <5 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %28 = load <6 x float>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %29 = load <7 x float>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %30 = load <9 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %31 = load <10 x float>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %32 = load <11 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %33 = load <12 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %34 = load <13 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %35 = load <14 x float>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %36 = load <15 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %39 = load <3 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %41 = load <5 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %43 = load <7 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %44 = load <9 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %45 = load <10 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %46 = load <11 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %45 = load <10 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %47 = load <12 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %48 = load <13 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %49 = load <14 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %50 = load <15 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %50 = load <15 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %51 = load <17 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %52 = load <18 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %53 = load <19 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %53 = load <19 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %54 = load <20 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %55 = load <21 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %57 = load <23 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %55 = load <21 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %56 = load <22 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %57 = load <23 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %58 = load <24 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %59 = load <25 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %60 = load <26 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %61 = load <27 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %59 = load <25 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %63 = load <29 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %65 = load <31 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %64 = load <30 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %65 = load <31 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %67 = load <2 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %68 = load <3 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %68 = load <3 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %70 = load <5 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %71 = load <6 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %72 = load <7 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %74 = load <9 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %75 = load <10 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %76 = load <11 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %78 = load <13 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %79 = load <14 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %80 = load <15 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %81 = load <17 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %82 = load <18 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %83 = load <19 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %84 = load <20 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %85 = load <21 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %86 = load <22 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %87 = load <23 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %80 = load <15 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %84 = load <20 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %87 = load <23 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %88 = load <24 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %89 = load <25 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %90 = load <26 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %91 = load <27 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %92 = load <28 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %93 = load <29 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %94 = load <30 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %95 = load <31 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %91 = load <27 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %93 = load <29 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %94 = load <30 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %95 = load <31 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %96 = load <33 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %97 = load <34 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %98 = load <35 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %98 = load <35 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %99 = load <36 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %100 = load <37 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %101 = load <38 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %102 = load <39 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %100 = load <37 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %101 = load <38 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %102 = load <39 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %103 = load <40 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %104 = load <41 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %105 = load <42 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %106 = load <43 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %108 = load <45 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %109 = load <46 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %110 = load <47 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %104 = load <41 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %105 = load <42 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %106 = load <43 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %107 = load <44 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %108 = load <45 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %109 = load <46 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %110 = load <47 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %111 = load <48 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %112 = load <49 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %113 = load <50 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %114 = load <51 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %115 = load <52 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %116 = load <53 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %117 = load <54 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %118 = load <55 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %112 = load <49 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %113 = load <50 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %114 = load <51 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %116 = load <53 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %118 = load <55 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %120 = load <57 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %121 = load <58 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %122 = load <59 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %124 = load <61 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %125 = load <62 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %126 = load <63 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %120 = load <57 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %123 = load <60 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %124 = load <61 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %125 = load <62 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %126 = load <63 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %130 = load <7 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %131 = load <9 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %132 = load <10 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %133 = load <11 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %134 = load <12 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %135 = load <13 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %136 = load <14 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %137 = load <15 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %138 = load <17 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %139 = load <18 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %140 = load <19 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %141 = load <20 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %142 = load <21 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %143 = load <22 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %144 = load <23 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %145 = load <24 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %146 = load <25 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %147 = load <26 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %148 = load <27 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %149 = load <28 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %150 = load <29 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %151 = load <30 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %152 = load <31 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %131 = load <9 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %132 = load <10 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %133 = load <11 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %134 = load <12 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %135 = load <13 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %136 = load <14 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %137 = load <15 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %138 = load <17 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %139 = load <18 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %140 = load <19 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %141 = load <20 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %142 = load <21 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %143 = load <22 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %144 = load <23 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %145 = load <24 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %146 = load <25 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %147 = load <26 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %148 = load <27 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %149 = load <28 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %150 = load <29 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %151 = load <30 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %152 = load <31 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %153 = load <33 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %154 = load <34 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %155 = load <35 x i1>, ptr undef, align 1
@@ -5802,29 +5802,29 @@ define i32 @loads_partial_align1(i32 %arg) {
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %158 = load <38 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %159 = load <39 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %160 = load <40 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %161 = load <41 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %162 = load <42 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %163 = load <43 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %164 = load <44 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %165 = load <45 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %166 = load <46 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %167 = load <47 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %161 = load <41 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %162 = load <42 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %163 = load <43 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %164 = load <44 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %165 = load <45 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %166 = load <46 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %167 = load <47 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %168 = load <48 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %169 = load <49 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %170 = load <50 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %171 = load <51 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %172 = load <52 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %173 = load <53 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %174 = load <54 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %175 = load <55 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %176 = load <56 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %177 = load <57 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %178 = load <58 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %179 = load <59 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %180 = load <60 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %181 = load <61 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %182 = load <62 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %183 = load <63 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %169 = load <49 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %170 = load <50 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %171 = load <51 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %172 = load <52 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %173 = load <53 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %174 = load <54 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %175 = load <55 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %176 = load <56 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %177 = load <57 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %178 = load <58 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %179 = load <59 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %180 = load <60 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %181 = load <61 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %182 = load <62 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %183 = load <63 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
; AVX512-LABEL: 'loads_partial_align1'
@@ -5840,177 +5840,177 @@ define i32 @loads_partial_align1(i32 %arg) {
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %10 = load <7 x double>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %14 = load <5 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %14 = load <5 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %15 = load <6 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %16 = load <7 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %17 = load <9 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %17 = load <9 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %18 = load <10 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %19 = load <11 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %20 = load <12 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %21 = load <13 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %21 = load <13 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %22 = load <14 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %23 = load <15 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %27 = load <5 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %28 = load <6 x float>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %29 = load <7 x float>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %30 = load <9 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %31 = load <10 x float>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %32 = load <11 x float>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %33 = load <12 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %34 = load <13 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %35 = load <14 x float>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %36 = load <15 x float>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %39 = load <3 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %41 = load <5 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %43 = load <7 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %44 = load <9 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %45 = load <10 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %46 = load <11 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %45 = load <10 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %47 = load <12 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %48 = load <13 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %49 = load <14 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %50 = load <15 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %51 = load <17 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %52 = load <18 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %53 = load <19 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %50 = load <15 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %51 = load <17 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %55 = load <21 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %56 = load <22 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %57 = load <23 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %57 = load <23 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %59 = load <25 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %60 = load <26 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %61 = load <27 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %60 = load <26 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %61 = load <27 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %62 = load <28 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %63 = load <29 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %64 = load <30 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %65 = load <31 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %63 = load <29 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %65 = load <31 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %67 = load <2 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %68 = load <3 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %68 = load <3 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %70 = load <5 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %71 = load <6 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %72 = load <7 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %74 = load <9 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %75 = load <10 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %76 = load <11 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %78 = load <13 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %79 = load <14 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %80 = load <15 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %81 = load <17 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %82 = load <18 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %83 = load <19 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %84 = load <20 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %85 = load <21 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %86 = load <22 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %87 = load <23 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %80 = load <15 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %84 = load <20 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %87 = load <23 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %88 = load <24 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %89 = load <25 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %90 = load <26 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %91 = load <27 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %92 = load <28 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %93 = load <29 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %94 = load <30 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %95 = load <31 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %96 = load <33 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %97 = load <34 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %98 = load <35 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %99 = load <36 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %100 = load <37 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %101 = load <38 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %102 = load <39 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %91 = load <27 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %93 = load <29 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %94 = load <30 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %95 = load <31 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %96 = load <33 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %97 = load <34 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %98 = load <35 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %102 = load <39 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %104 = load <41 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %105 = load <42 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %106 = load <43 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %107 = load <44 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %108 = load <45 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %109 = load <46 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %110 = load <47 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %106 = load <43 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %108 = load <45 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %109 = load <46 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %110 = load <47 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %112 = load <49 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %113 = load <50 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %114 = load <51 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %115 = load <52 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %116 = load <53 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %117 = load <54 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %118 = load <55 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %114 = load <51 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %115 = load <52 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %116 = load <53 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %117 = load <54 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %118 = load <55 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %119 = load <56 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %120 = load <57 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %121 = load <58 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %122 = load <59 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %123 = load <60 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %124 = load <61 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %125 = load <62 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 13 for instruction: %126 = load <63 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %120 = load <57 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %121 = load <58 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %122 = load <59 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %124 = load <61 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %125 = load <62 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %126 = load <63 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %130 = load <7 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %131 = load <9 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %132 = load <10 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %133 = load <11 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %134 = load <12 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %135 = load <13 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %136 = load <14 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %137 = load <15 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %138 = load <17 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %139 = load <18 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %140 = load <19 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %141 = load <20 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %142 = load <21 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %143 = load <22 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %144 = load <23 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %145 = load <24 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %146 = load <25 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %147 = load <26 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %148 = load <27 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %149 = load <28 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %150 = load <29 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %151 = load <30 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %152 = load <31 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %153 = load <33 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %154 = load <34 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %155 = load <35 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %156 = load <36 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %157 = load <37 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %158 = load <38 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %159 = load <39 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %160 = load <40 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %161 = load <41 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %162 = load <42 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %163 = load <43 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %164 = load <44 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %165 = load <45 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %166 = load <46 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %167 = load <47 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %168 = load <48 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %169 = load <49 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %170 = load <50 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %171 = load <51 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %172 = load <52 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %173 = load <53 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %174 = load <54 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %175 = load <55 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %176 = load <56 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %177 = load <57 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %178 = load <58 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %179 = load <59 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %180 = load <60 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %181 = load <61 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %182 = load <62 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %183 = load <63 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %131 = load <9 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %132 = load <10 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %133 = load <11 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %134 = load <12 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %135 = load <13 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %136 = load <14 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %137 = load <15 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %138 = load <17 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %139 = load <18 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %140 = load <19 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %141 = load <20 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %142 = load <21 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %143 = load <22 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %144 = load <23 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %145 = load <24 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %146 = load <25 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %147 = load <26 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %148 = load <27 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %149 = load <28 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %150 = load <29 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %151 = load <30 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %152 = load <31 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %153 = load <33 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %154 = load <34 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %155 = load <35 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %156 = load <36 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %157 = load <37 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %158 = load <38 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %159 = load <39 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %160 = load <40 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %161 = load <41 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %162 = load <42 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %163 = load <43 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %164 = load <44 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %165 = load <45 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %166 = load <46 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %167 = load <47 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %168 = load <48 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %169 = load <49 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %170 = load <50 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %171 = load <51 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %172 = load <52 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %173 = load <53 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %174 = load <54 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %175 = load <55 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %176 = load <56 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %177 = load <57 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %178 = load <58 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %179 = load <59 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %180 = load <60 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %181 = load <61 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %182 = load <62 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %183 = load <63 x i1>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
load <1 x i64>, ptr undef, align 1
diff --git a/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll b/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
new file mode 100644
index 0000000000000..e6a79f13dfa68
--- /dev/null
+++ b/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
@@ -0,0 +1,68 @@
+; NOTE: Assertions have been autogenerated by utils/update_analyze_test_checks.py
+; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+sse2 | FileCheck %s --check-prefixes=SSE2
+; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+sse4.1 | FileCheck %s --check-prefixes=SSE41
+; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+avx2 | FileCheck %s --check-prefixes=AVX2
+
+; Non-power-of-2 sub-16B vector loads are materialized by a base load plus a
+; folded vpinsr*(mem) per remaining in-lane chunk (1B->pinsrb, 2B->pinsrw,
+; 4B->pinsrd), i.e. one instruction per pow2 chunk. The cost must reflect that
+; (popcount(VF) loads for in-lane types, plus real vinserti128 stitches), not
+; double-count the folded insert. The fold is ISA-gated: the byte (pinsrb) and
+; dword (pinsrd) folds need SSE4.1, the word (pinsrw) fold only needs SSE2, so
+; on SSE2 <3 x i8> stays high while <3 x i16> already gets the relief. The same
+; applies to FP (the 4-byte float remainder folds via INSERTPS/SSE4.1), while
+; 8-byte chunks (<3 x i64>/<3 x double>) are outside the fold set and unchanged.
+; Power-of-2 loads (e.g. <2 x i8>) are left as-is.
+
+define void @npot_loads(ptr %p) {
+; SSE2-LABEL: 'npot_loads'
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %v3i8 = load <3 x i8>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %v7i8 = load <7 x i8>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i16 = load <3 x i16>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %v3i32 = load <3 x i32>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %v7i32 = load <7 x i32>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i64 = load <3 x i64>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %v3f32 = load <3 x float>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
+;
+; SSE41-LABEL: 'npot_loads'
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i8 = load <3 x i8>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7i8 = load <7 x i8>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i16 = load <3 x i16>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i32 = load <3 x i32>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7i32 = load <7 x i32>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i64 = load <3 x i64>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f32 = load <3 x float>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
+;
+; AVX2-LABEL: 'npot_loads'
+; AVX2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i8 = load <3 x i8>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7i8 = load <7 x i8>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i16 = load <3 x i16>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i32 = load <3 x i32>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7i32 = load <7 x i32>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i64 = load <3 x i64>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f32 = load <3 x float>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
+;
+ %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+ %v3i8 = load <3 x i8>, ptr %p, align 1
+ %v7i8 = load <7 x i8>, ptr %p, align 1
+ %v3i16 = load <3 x i16>, ptr %p, align 1
+ %v3i32 = load <3 x i32>, ptr %p, align 1
+ %v7i32 = load <7 x i32>, ptr %p, align 1
+ %v3i64 = load <3 x i64>, ptr %p, align 1
+ %v3f32 = load <3 x float>, ptr %p, align 1
+ %v7f32 = load <7 x float>, ptr %p, align 1
+ %v3f64 = load <3 x double>, ptr %p, align 1
+ ret void
+}
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-3.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-3.ll
index 15baa2a627f23..156bd432a3447 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-3.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-3.ll
@@ -30,7 +30,7 @@ define void @test() {
;
; AVX2-LABEL: 'test'
; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i16, ptr %in0, align 2
-; AVX2: Cost of 8 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX2: Cost of 7 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX2: ir<%v0> = load from index 0
; AVX2: ir<%v1> = load from index 1
; AVX2: ir<%v2> = load from index 2
@@ -53,7 +53,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i16, ptr %in0, align 2
-; AVX512DQ: Cost of 8 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX512DQ: Cost of 7 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-5.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-5.ll
index 66714ab44b62a..a7b6bf588af09 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-5.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-5.ll
@@ -38,7 +38,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i16, ptr %in0, align 2
-; AVX512DQ: Cost of 25 for VF 2: INTERLEAVE-GROUP with factor 5, ir<%in0>
+; AVX512DQ: Cost of 24 for VF 2: INTERLEAVE-GROUP with factor 5, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-7.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-7.ll
index dcbf02f16f68d..26e8279763425 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-7.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-7.ll
@@ -38,7 +38,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i16, ptr %in0, align 2
-; AVX512DQ: Cost of 34 for VF 2: INTERLEAVE-GROUP with factor 7, ir<%in0>
+; AVX512DQ: Cost of 33 for VF 2: INTERLEAVE-GROUP with factor 7, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-3.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-3.ll
index e3224b8938f7c..f385a46789ecc 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-3.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-3.ll
@@ -30,11 +30,11 @@ define void @test() {
;
; AVX2-LABEL: 'test'
; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX2: Cost of 7 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX2: Cost of 5 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX2: ir<%v0> = load from index 0
; AVX2: ir<%v1> = load from index 1
; AVX2: ir<%v2> = load from index 2
-; AVX2: Cost of 6 for VF 4: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX2: Cost of 5 for VF 4: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX2: ir<%v0> = load from index 0
; AVX2: ir<%v1> = load from index 1
; AVX2: ir<%v2> = load from index 2
@@ -53,11 +53,11 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX512DQ: Cost of 7 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX512DQ: Cost of 5 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
-; AVX512DQ: Cost of 6 for VF 4: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX512DQ: Cost of 5 for VF 4: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-5.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-5.ll
index 3d887cb7958a9..490ac46b0b4ac 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-5.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-5.ll
@@ -38,13 +38,13 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX512DQ: Cost of 24 for VF 2: INTERLEAVE-GROUP with factor 5, ir<%in0>
+; AVX512DQ: Cost of 22 for VF 2: INTERLEAVE-GROUP with factor 5, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
; AVX512DQ: ir<%v3> = load from index 3
; AVX512DQ: ir<%v4> = load from index 4
-; AVX512DQ: Cost of 45 for VF 4: INTERLEAVE-GROUP with factor 5, ir<%in0>
+; AVX512DQ: Cost of 44 for VF 4: INTERLEAVE-GROUP with factor 5, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-6.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-6.ll
index a2f22da5d7a4d..08c5f31fbd5d0 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-6.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-6.ll
@@ -30,7 +30,7 @@ define void @test() {
;
; AVX2-LABEL: 'test'
; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX2: Cost of 9 for VF 2: INTERLEAVE-GROUP with factor 6, ir<%in0>
+; AVX2: Cost of 8 for VF 2: INTERLEAVE-GROUP with factor 6, ir<%in0>
; AVX2: ir<%v0> = load from index 0
; AVX2: ir<%v1> = load from index 1
; AVX2: ir<%v2> = load from index 2
@@ -68,7 +68,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX512DQ: Cost of 9 for VF 2: INTERLEAVE-GROUP with factor 6, ir<%in0>
+; AVX512DQ: Cost of 8 for VF 2: INTERLEAVE-GROUP with factor 6, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-7.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-7.ll
index d9d66e5b89598..afbe3c05bfb9e 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-7.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-7.ll
@@ -38,7 +38,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX512DQ: Cost of 34 for VF 2: INTERLEAVE-GROUP with factor 7, ir<%in0>
+; AVX512DQ: Cost of 31 for VF 2: INTERLEAVE-GROUP with factor 7, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
@@ -46,7 +46,7 @@ define void @test() {
; AVX512DQ: ir<%v4> = load from index 4
; AVX512DQ: ir<%v5> = load from index 5
; AVX512DQ: ir<%v6> = load from index 6
-; AVX512DQ: Cost of 62 for VF 4: INTERLEAVE-GROUP with factor 7, ir<%in0>
+; AVX512DQ: Cost of 61 for VF 4: INTERLEAVE-GROUP with factor 7, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/SLPVectorizer/X86/nonpow2-shared-weight-reduction.ll b/llvm/test/Transforms/SLPVectorizer/X86/nonpow2-shared-weight-reduction.ll
new file mode 100644
index 0000000000000..7698e492458c8
--- /dev/null
+++ b/llvm/test/Transforms/SLPVectorizer/X86/nonpow2-shared-weight-reduction.ll
@@ -0,0 +1,150 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -passes=slp-vectorizer -mtriple=x86_64-unknown-linux-gnu -mcpu=x86-64-v3 \
+; RUN: -slp-vectorize-non-power-of-2=true -S < %s | FileCheck %s --check-prefixes=NPOT
+; RUN: opt -passes=slp-vectorizer -mtriple=x86_64-unknown-linux-gnu -mcpu=x86-64-v3 \
+; RUN: -slp-vectorize-non-power-of-2=false -S < %s | FileCheck %s --check-prefixes=SCALAR
+
+; Three independent reductions over contiguous RGB bytes sharing a
+; single broadcast weight. SLP forms a width-3 tree, but at baseline the
+; non-power-of-2 <3 x i8> load is over-priced (the folded vpinsrb(mem) is
+; double-counted), so the tree cost is 0 and the tree is discarded to scalar.
+; Once the load is priced correctly the tree cost goes negative and the width-3
+; form is kept (vpmovzxbd + vpbroadcastd + vpmulld + vpaddd).
+
+define void @ship3(ptr noundef readonly %weights, ptr noundef readonly %src, i64 noundef %bpp, ptr noundef writeonly %dst, i32 noundef %n) {
+; NPOT-LABEL: @ship3(
+; NPOT-NEXT: [[N_EXT:%.*]] = sext i32 [[N:%.*]] to i64
+; NPOT-NEXT: [[POS:%.*]] = icmp sgt i32 [[N]], 0
+; NPOT-NEXT: br i1 [[POS]], label [[LOOP:%.*]], label [[EXIT:%.*]]
+; NPOT: loop:
+; NPOT-NEXT: [[IV:%.*]] = phi i64 [ 0, [[TMP0:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
+; NPOT-NEXT: [[TMP1:%.*]] = phi <3 x i32> [ zeroinitializer, [[TMP0]] ], [ [[TMP7:%.*]], [[LOOP]] ]
+; NPOT-NEXT: [[WGEP:%.*]] = getelementptr inbounds i32, ptr [[WEIGHTS:%.*]], i64 [[IV]]
+; NPOT-NEXT: [[W:%.*]] = load i32, ptr [[WGEP]], align 4
+; NPOT-NEXT: [[OFF:%.*]] = mul nsw i64 [[IV]], [[BPP:%.*]]
+; NPOT-NEXT: [[P0:%.*]] = getelementptr inbounds i8, ptr [[SRC:%.*]], i64 [[OFF]]
+; NPOT-NEXT: [[TMP2:%.*]] = load <3 x i8>, ptr [[P0]], align 1
+; NPOT-NEXT: [[TMP3:%.*]] = zext <3 x i8> [[TMP2]] to <3 x i32>
+; NPOT-NEXT: [[TMP4:%.*]] = insertelement <3 x i32> poison, i32 [[W]], i64 0
+; NPOT-NEXT: [[TMP5:%.*]] = shufflevector <3 x i32> [[TMP4]], <3 x i32> poison, <3 x i32> zeroinitializer
+; NPOT-NEXT: [[TMP6:%.*]] = mul nsw <3 x i32> [[TMP5]], [[TMP3]]
+; NPOT-NEXT: [[TMP7]] = add nsw <3 x i32> [[TMP6]], [[TMP1]]
+; NPOT-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1
+; NPOT-NEXT: [[DONE:%.*]] = icmp eq i64 [[IV_NEXT]], [[N_EXT]]
+; NPOT-NEXT: br i1 [[DONE]], label [[EXIT_LOOP:%.*]], label [[LOOP]]
+; NPOT: exit.loop:
+; NPOT-NEXT: [[TMP8:%.*]] = lshr <3 x i32> [[TMP7]], splat (i32 16)
+; NPOT-NEXT: [[TMP9:%.*]] = trunc <3 x i32> [[TMP8]] to <3 x i8>
+; NPOT-NEXT: br label [[EXIT]]
+; NPOT: exit:
+; NPOT-NEXT: [[TMP10:%.*]] = phi <3 x i8> [ zeroinitializer, [[TMP0]] ], [ [[TMP9]], [[EXIT_LOOP]] ]
+; NPOT-NEXT: [[D1:%.*]] = getelementptr inbounds i8, ptr [[DST:%.*]], i64 1
+; NPOT-NEXT: [[D2:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 2
+; NPOT-NEXT: [[TMP11:%.*]] = extractelement <3 x i8> [[TMP10]], i64 0
+; NPOT-NEXT: store i8 [[TMP11]], ptr [[DST]], align 1
+; NPOT-NEXT: [[TMP12:%.*]] = extractelement <3 x i8> [[TMP10]], i64 1
+; NPOT-NEXT: store i8 [[TMP12]], ptr [[D1]], align 1
+; NPOT-NEXT: [[TMP13:%.*]] = extractelement <3 x i8> [[TMP10]], i64 2
+; NPOT-NEXT: store i8 [[TMP13]], ptr [[D2]], align 1
+; NPOT-NEXT: ret void
+;
+; SCALAR-LABEL: @ship3(
+; SCALAR-NEXT: [[N_EXT:%.*]] = sext i32 [[N:%.*]] to i64
+; SCALAR-NEXT: [[POS:%.*]] = icmp sgt i32 [[N]], 0
+; SCALAR-NEXT: br i1 [[POS]], label [[LOOP:%.*]], label [[EXIT:%.*]]
+; SCALAR: loop:
+; SCALAR-NEXT: [[IV:%.*]] = phi i64 [ 0, [[TMP0:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
+; SCALAR-NEXT: [[ACCB:%.*]] = phi i32 [ 0, [[TMP0]] ], [ [[ADDB:%.*]], [[LOOP]] ]
+; SCALAR-NEXT: [[ACCG:%.*]] = phi i32 [ 0, [[TMP0]] ], [ [[ADDG:%.*]], [[LOOP]] ]
+; SCALAR-NEXT: [[ACCR:%.*]] = phi i32 [ 0, [[TMP0]] ], [ [[ADDR:%.*]], [[LOOP]] ]
+; SCALAR-NEXT: [[WGEP:%.*]] = getelementptr inbounds i32, ptr [[WEIGHTS:%.*]], i64 [[IV]]
+; SCALAR-NEXT: [[W:%.*]] = load i32, ptr [[WGEP]], align 4
+; SCALAR-NEXT: [[OFF:%.*]] = mul nsw i64 [[IV]], [[BPP:%.*]]
+; SCALAR-NEXT: [[P0:%.*]] = getelementptr inbounds i8, ptr [[SRC:%.*]], i64 [[OFF]]
+; SCALAR-NEXT: [[P1:%.*]] = getelementptr inbounds i8, ptr [[P0]], i64 1
+; SCALAR-NEXT: [[P2:%.*]] = getelementptr inbounds i8, ptr [[P0]], i64 2
+; SCALAR-NEXT: [[B:%.*]] = load i8, ptr [[P0]], align 1
+; SCALAR-NEXT: [[G:%.*]] = load i8, ptr [[P1]], align 1
+; SCALAR-NEXT: [[R:%.*]] = load i8, ptr [[P2]], align 1
+; SCALAR-NEXT: [[ZB:%.*]] = zext i8 [[B]] to i32
+; SCALAR-NEXT: [[ZG:%.*]] = zext i8 [[G]] to i32
+; SCALAR-NEXT: [[ZR:%.*]] = zext i8 [[R]] to i32
+; SCALAR-NEXT: [[MB:%.*]] = mul nsw i32 [[W]], [[ZB]]
+; SCALAR-NEXT: [[MG:%.*]] = mul nsw i32 [[W]], [[ZG]]
+; SCALAR-NEXT: [[MR:%.*]] = mul nsw i32 [[W]], [[ZR]]
+; SCALAR-NEXT: [[ADDB]] = add nsw i32 [[MB]], [[ACCB]]
+; SCALAR-NEXT: [[ADDG]] = add nsw i32 [[MG]], [[ACCG]]
+; SCALAR-NEXT: [[ADDR]] = add nsw i32 [[MR]], [[ACCR]]
+; SCALAR-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1
+; SCALAR-NEXT: [[DONE:%.*]] = icmp eq i64 [[IV_NEXT]], [[N_EXT]]
+; SCALAR-NEXT: br i1 [[DONE]], label [[EXIT_LOOP:%.*]], label [[LOOP]]
+; SCALAR: exit.loop:
+; SCALAR-NEXT: [[SB:%.*]] = lshr i32 [[ADDB]], 16
+; SCALAR-NEXT: [[SG:%.*]] = lshr i32 [[ADDG]], 16
+; SCALAR-NEXT: [[SR:%.*]] = lshr i32 [[ADDR]], 16
+; SCALAR-NEXT: [[TB:%.*]] = trunc i32 [[SB]] to i8
+; SCALAR-NEXT: [[TG:%.*]] = trunc i32 [[SG]] to i8
+; SCALAR-NEXT: [[TR:%.*]] = trunc i32 [[SR]] to i8
+; SCALAR-NEXT: br label [[EXIT]]
+; SCALAR: exit:
+; SCALAR-NEXT: [[OB:%.*]] = phi i8 [ 0, [[TMP0]] ], [ [[TB]], [[EXIT_LOOP]] ]
+; SCALAR-NEXT: [[OG:%.*]] = phi i8 [ 0, [[TMP0]] ], [ [[TG]], [[EXIT_LOOP]] ]
+; SCALAR-NEXT: [[OR:%.*]] = phi i8 [ 0, [[TMP0]] ], [ [[TR]], [[EXIT_LOOP]] ]
+; SCALAR-NEXT: [[D1:%.*]] = getelementptr inbounds i8, ptr [[DST:%.*]], i64 1
+; SCALAR-NEXT: [[D2:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 2
+; SCALAR-NEXT: store i8 [[OB]], ptr [[DST]], align 1
+; SCALAR-NEXT: store i8 [[OG]], ptr [[D1]], align 1
+; SCALAR-NEXT: store i8 [[OR]], ptr [[D2]], align 1
+; SCALAR-NEXT: ret void
+;
+ %n.ext = sext i32 %n to i64
+ %pos = icmp sgt i32 %n, 0
+ br i1 %pos, label %loop, label %exit
+
+loop:
+ %iv = phi i64 [ 0, %0 ], [ %iv.next, %loop ]
+ %accb = phi i32 [ 0, %0 ], [ %addb, %loop ]
+ %accg = phi i32 [ 0, %0 ], [ %addg, %loop ]
+ %accr = phi i32 [ 0, %0 ], [ %addr, %loop ]
+ %wgep = getelementptr inbounds i32, ptr %weights, i64 %iv
+ %w = load i32, ptr %wgep, align 4
+ %off = mul nsw i64 %iv, %bpp
+ %p0 = getelementptr inbounds i8, ptr %src, i64 %off
+ %p1 = getelementptr inbounds i8, ptr %p0, i64 1
+ %p2 = getelementptr inbounds i8, ptr %p0, i64 2
+ %b = load i8, ptr %p0, align 1
+ %g = load i8, ptr %p1, align 1
+ %r = load i8, ptr %p2, align 1
+ %zb = zext i8 %b to i32
+ %zg = zext i8 %g to i32
+ %zr = zext i8 %r to i32
+ %mb = mul nsw i32 %w, %zb
+ %mg = mul nsw i32 %w, %zg
+ %mr = mul nsw i32 %w, %zr
+ %addb = add nsw i32 %mb, %accb
+ %addg = add nsw i32 %mg, %accg
+ %addr = add nsw i32 %mr, %accr
+ %iv.next = add nuw nsw i64 %iv, 1
+ %done = icmp eq i64 %iv.next, %n.ext
+ br i1 %done, label %exit.loop, label %loop
+
+exit.loop:
+ %sb = lshr i32 %addb, 16
+ %sg = lshr i32 %addg, 16
+ %sr = lshr i32 %addr, 16
+ %tb = trunc i32 %sb to i8
+ %tg = trunc i32 %sg to i8
+ %tr = trunc i32 %sr to i8
+ br label %exit
+
+exit:
+ %ob = phi i8 [ 0, %0 ], [ %tb, %exit.loop ]
+ %og = phi i8 [ 0, %0 ], [ %tg, %exit.loop ]
+ %or = phi i8 [ 0, %0 ], [ %tr, %exit.loop ]
+ %d1 = getelementptr inbounds i8, ptr %dst, i64 1
+ %d2 = getelementptr inbounds i8, ptr %dst, i64 2
+ store i8 %ob, ptr %dst, align 1
+ store i8 %og, ptr %d1, align 1
+ store i8 %or, ptr %d2, align 1
+ ret void
+}
>From 6af3a2afdb54b5abbfddb785b5c211856bc24620 Mon Sep 17 00:00:00 2001
From: mbhade <mbhade at amd.com>
Date: Tue, 1 Sep 2026 12:48:53 +0530
Subject: [PATCH 2/3] [X86] NFC: Format folded insert cost logic
Apply clang-format to the folded load-insert cost calculation.
Co-authored-by: Cursor <cursoragent at cursor.com>
---
llvm/lib/Target/X86/X86TargetTransformInfo.cpp | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index 36b01e3ef9859..fbe7936f06685 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -5558,11 +5558,10 @@ InstructionCost X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
// below must be skipped - otherwise the folded insert is double-counted.
// Stores (the symmetric PEXTR*(mem) fold) are left unchanged here.
bool Is0thSubVec = (NumEltDone() % LT.second.getVectorNumElements()) == 0;
- bool FoldedInLaneInsert =
- IsLoad && !Is0thSubVec &&
- ((CurrOpSizeBytes == 1 && ST->hasSSE41()) ||
- (CurrOpSizeBytes == 2 && ST->hasSSE2()) ||
- (CurrOpSizeBytes == 4 && ST->hasSSE41()));
+ bool FoldedInLaneInsert = IsLoad && !Is0thSubVec &&
+ ((CurrOpSizeBytes == 1 && ST->hasSSE41()) ||
+ (CurrOpSizeBytes == 2 && ST->hasSSE2()) ||
+ (CurrOpSizeBytes == 4 && ST->hasSSE41()));
if (CurrOpSizeBytes == 32 && ST->isUnalignedMem32Slow())
Cost += 2;
else if (CurrOpSizeBytes < 4 && !FoldedInLaneInsert)
>From e12a444e29353548a6e43c1d483821ff6cb69cec Mon Sep 17 00:00:00 2001
From: mbhade <mbhade at amd.com>
Date: Wed, 9 Sep 2026 10:54:07 +0530
Subject: [PATCH 3/3] [X86] Add AVX512 coverage to folded-load cost test
Extend nonpow2-load-fold.ll with an -mattr=+avx512f,+avx512vl RUN line so
the corrected non-power-of-2 folded-insert load costs are checked at the
AVX512 ISA level as well.
Co-authored-by: Cursor <cursoragent at cursor.com>
---
.../Analysis/CostModel/X86/nonpow2-load-fold.ll | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll b/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
index e6a79f13dfa68..abd529be18b7b 100644
--- a/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
+++ b/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
@@ -2,6 +2,7 @@
; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+sse2 | FileCheck %s --check-prefixes=SSE2
; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+sse4.1 | FileCheck %s --check-prefixes=SSE41
; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+avx2 | FileCheck %s --check-prefixes=AVX2
+; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+avx512f,+avx512vl | FileCheck %s --check-prefixes=AVX512
; Non-power-of-2 sub-16B vector loads are materialized by a base load plus a
; folded vpinsr*(mem) per remaining in-lane chunk (1B->pinsrb, 2B->pinsrw,
@@ -53,6 +54,19 @@ define void @npot_loads(ptr %p) {
; AVX2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
; AVX2-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
+;
+; AVX512-LABEL: 'npot_loads'
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i8 = load <3 x i8>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7i8 = load <7 x i8>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i16 = load <3 x i16>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i32 = load <3 x i32>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7i32 = load <7 x i32>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i64 = load <3 x i64>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f32 = load <3 x float>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
%pow2_v2i8 = load <2 x i8>, ptr %p, align 1
%v3i8 = load <3 x i8>, ptr %p, align 1
More information about the llvm-commits
mailing list