[llvm] [X86] Correct folded insert costs for partial vector loads (PR #220174)
Simon Pilgrim via llvm-commits
llvm-commits at lists.llvm.org
Tue Sep 22 01:24:53 PDT 2026
https://github.com/RKSimon updated https://github.com/llvm/llvm-project/pull/220174
>From 1ecdfa9860c80357a78197a515f15d8b917a8b78 Mon Sep 17 00:00:00 2001
From: mbhade <mbhade at amd.com>
Date: Tue, 25 Aug 2026 14:35:13 +0530
Subject: [PATCH 1/3] [X86] Correct folded insert costs for partial vector
loads
Recent SLP support for non-power-of-two nodes can form width-3 trees and relies on target costs to determine profitability:
https://github.com/llvm/llvm-project/pull/194189
X86 getMemoryOpCost counted a non-0th 1/2/4-byte PINSR*(mem) chunk as both a memory operation and a separate lane insertion. This overpricing caused SLP to reject the width-3 shared-weight RGB reduction.
Recognize non-0th folded load-insert chunks and charge each once. Leading sub-dword loads, stores, and 8-byte chunks remain unchanged.
---
.../lib/Target/X86/X86TargetTransformInfo.cpp | 21 +-
.../test/Analysis/CostModel/X86/load_store.ll | 1306 ++++++++---------
.../CostModel/X86/nonpow2-load-fold.ll | 68 +
.../interleaved-load-i16-stride-3.ll | 4 +-
.../interleaved-load-i16-stride-5.ll | 2 +-
.../interleaved-load-i16-stride-7.ll | 2 +-
.../CostModel/interleaved-load-i8-stride-3.ll | 8 +-
.../CostModel/interleaved-load-i8-stride-5.ll | 4 +-
.../CostModel/interleaved-load-i8-stride-6.ll | 4 +-
.../CostModel/interleaved-load-i8-stride-7.ll | 4 +-
.../X86/nonpow2-shared-weight-reduction.ll | 39 +-
11 files changed, 765 insertions(+), 697 deletions(-)
create mode 100644 llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index fa69ad9f05603f..e30cceec912cda 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -5896,9 +5896,24 @@ InstructionCost X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
// Sandybridge.
// Sub-32-bit loads/stores will be slower either with PINSR*/PEXTR* or
// will be scalarized.
+ //
+ // For a vector load, each non-0th 1/2/4-byte in-lane remainder chunk is
+ // materialized by a *single* folded PINSR*(mem) that both loads and
+ // inserts the lane (1B->PINSRB, 2B->PINSRW, 4B->PINSRD; the byte and
+ // dword folds need SSE4.1, the word fold only SSE2).
+ // When that fold is available the chunk is one instruction, so it must be
+ // priced once here (as a plain load) and the separate lane-insert charge
+ // below must be skipped - otherwise the folded insert is double-counted.
+ // Stores (the symmetric PEXTR*(mem) fold) are left unchanged here.
+ bool Is0thSubVec = (NumEltDone() % LT.second.getVectorNumElements()) == 0;
+ bool FoldedInLaneInsert =
+ IsLoad && !Is0thSubVec &&
+ ((CurrOpSizeBytes == 1 && ST->hasSSE41()) ||
+ (CurrOpSizeBytes == 2 && ST->hasSSE2()) ||
+ (CurrOpSizeBytes == 4 && ST->hasSSE41()));
if (CurrOpSizeBytes == 32 && ST->isUnalignedMem32Slow())
Cost += 2;
- else if (CurrOpSizeBytes < 4)
+ else if (CurrOpSizeBytes < 4 && !FoldedInLaneInsert)
Cost += 2;
else
Cost += 1;
@@ -5908,8 +5923,6 @@ InstructionCost X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
if (IsLoad && OpInfo.isUniform())
return Cost;
- bool Is0thSubVec = (NumEltDone() % LT.second.getVectorNumElements()) == 0;
-
// If we have fully processed the previous reg, we need to replenish it.
if (SubVecEltsLeft == 0) {
SubVecEltsLeft += CurrVecTy->getNumElements();
@@ -5925,7 +5938,7 @@ InstructionCost X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
// for smaller widths (32/16/8) we have to insert/extract them separately.
// Again, it's free for the 0'th subreg (if op is 32/64 bit wide,
// but let's pretend that it is also true for 16/8 bit wide ops...)
- if (CurrOpSizeBytes <= 32 / 8 && !Is0thSubVec) {
+ if (CurrOpSizeBytes <= 32 / 8 && !Is0thSubVec && !FoldedInLaneInsert) {
int NumEltDoneInCurrXMM = NumEltDone() % NumEltPerXMM;
assert(NumEltDoneInCurrXMM % CurrNumEltPerOp == 0 && "");
int CoalescedVecEltIdx = NumEltDoneInCurrXMM / CurrNumEltPerOp;
diff --git a/llvm/test/Analysis/CostModel/X86/load_store.ll b/llvm/test/Analysis/CostModel/X86/load_store.ll
index ef36bd70707242..0db96a06c578fa 100644
--- a/llvm/test/Analysis/CostModel/X86/load_store.ll
+++ b/llvm/test/Analysis/CostModel/X86/load_store.ll
@@ -4222,120 +4222,120 @@ define i32 @loads_partial_align4(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %10 = load <7 x double>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %14 = load <5 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %15 = load <6 x i32>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %16 = load <7 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %17 = load <9 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %18 = load <10 x i32>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %19 = load <11 x i32>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %20 = load <12 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %21 = load <13 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %22 = load <14 x i32>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %23 = load <15 x i32>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %27 = load <5 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %28 = load <6 x float>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %29 = load <7 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %30 = load <9 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %31 = load <10 x float>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %32 = load <11 x float>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %33 = load <12 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %34 = load <13 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %35 = load <14 x float>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %36 = load <15 x float>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %39 = load <3 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %41 = load <5 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %43 = load <7 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %44 = load <9 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %45 = load <10 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %46 = load <11 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %47 = load <12 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %50 = load <15 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %48 = load <13 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %49 = load <14 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %50 = load <15 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %51 = load <17 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %53 = load <19 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %55 = load <21 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %56 = load <22 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %57 = load <23 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %57 = load <23 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %59 = load <25 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %61 = load <27 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %63 = load <29 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %65 = load <31 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %64 = load <30 x i16>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %65 = load <31 x i16>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %67 = load <2 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %68 = load <3 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %70 = load <5 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %71 = load <6 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %72 = load <7 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %74 = load <9 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %75 = load <10 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %78 = load <13 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %79 = load <14 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %80 = load <15 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %76 = load <11 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %80 = load <15 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %81 = load <17 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %82 = load <18 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %83 = load <19 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %84 = load <20 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %87 = load <23 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %85 = load <21 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %86 = load <22 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %87 = load <23 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %88 = load <24 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %91 = load <27 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %93 = load <29 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %94 = load <30 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %95 = load <31 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %89 = load <25 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %90 = load <26 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %91 = load <27 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %92 = load <28 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %93 = load <29 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %94 = load <30 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %95 = load <31 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %96 = load <33 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %97 = load <34 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %98 = load <35 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %100 = load <37 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %101 = load <38 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %102 = load <39 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %102 = load <39 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %104 = load <41 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %105 = load <42 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %106 = load <43 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %107 = load <44 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %108 = load <45 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %109 = load <46 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %110 = load <47 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %106 = load <43 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %108 = load <45 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %109 = load <46 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %110 = load <47 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %112 = load <49 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %113 = load <50 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %114 = load <51 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %116 = load <53 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %117 = load <54 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %118 = load <55 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %116 = load <53 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %118 = load <55 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %120 = load <57 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %121 = load <58 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %124 = load <61 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %125 = load <62 x i8>, ptr undef, align 4
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %126 = load <63 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %120 = load <57 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %122 = load <59 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %123 = load <60 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %124 = load <61 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %125 = load <62 x i8>, ptr undef, align 4
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %126 = load <63 x i8>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %127 = load <3 x i1>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %128 = load <5 x i1>, ptr undef, align 4
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %129 = load <6 x i1>, ptr undef, align 4
@@ -4408,120 +4408,120 @@ define i32 @loads_partial_align4(i32 %arg) {
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %10 = load <7 x double>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %14 = load <5 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %14 = load <5 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %15 = load <6 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %16 = load <7 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %17 = load <9 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %18 = load <10 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %19 = load <11 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %20 = load <12 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %21 = load <13 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %21 = load <13 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %22 = load <14 x i32>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %23 = load <15 x i32>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %27 = load <5 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %28 = load <6 x float>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %29 = load <7 x float>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %30 = load <9 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %31 = load <10 x float>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %32 = load <11 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %33 = load <12 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %34 = load <13 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %35 = load <14 x float>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %36 = load <15 x float>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %39 = load <3 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %41 = load <5 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %43 = load <7 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %44 = load <9 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %45 = load <10 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %46 = load <11 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %45 = load <10 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %47 = load <12 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %48 = load <13 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %49 = load <14 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %50 = load <15 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %50 = load <15 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %51 = load <17 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %52 = load <18 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %53 = load <19 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %54 = load <20 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %57 = load <23 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %55 = load <21 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %56 = load <22 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %57 = load <23 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %58 = load <24 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %60 = load <26 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %61 = load <27 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %59 = load <25 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %63 = load <29 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %65 = load <31 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %64 = load <30 x i16>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %65 = load <31 x i16>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %67 = load <2 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %68 = load <3 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %70 = load <5 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %71 = load <6 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %72 = load <7 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %74 = load <9 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %75 = load <10 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %78 = load <13 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %79 = load <14 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %80 = load <15 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %81 = load <17 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %82 = load <18 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %84 = load <20 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %85 = load <21 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %86 = load <22 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %87 = load <23 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %76 = load <11 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %80 = load <15 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %83 = load <19 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %84 = load <20 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %87 = load <23 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %88 = load <24 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %89 = load <25 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %90 = load <26 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %91 = load <27 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %92 = load <28 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %93 = load <29 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %94 = load <30 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %95 = load <31 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %91 = load <27 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %93 = load <29 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %94 = load <30 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %95 = load <31 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %96 = load <33 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %97 = load <34 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %98 = load <35 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %99 = load <36 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %102 = load <39 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %100 = load <37 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %101 = load <38 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %102 = load <39 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %103 = load <40 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %106 = load <43 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %108 = load <45 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %109 = load <46 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %110 = load <47 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %104 = load <41 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %105 = load <42 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %106 = load <43 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %107 = load <44 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %108 = load <45 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %109 = load <46 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %110 = load <47 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %111 = load <48 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %114 = load <51 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %115 = load <52 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %116 = load <53 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %117 = load <54 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %118 = load <55 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %112 = load <49 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %113 = load <50 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %114 = load <51 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %116 = load <53 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %118 = load <55 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %120 = load <57 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %121 = load <58 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %124 = load <61 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %125 = load <62 x i8>, ptr undef, align 4
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %126 = load <63 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %120 = load <57 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %122 = load <59 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %123 = load <60 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %124 = load <61 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %125 = load <62 x i8>, ptr undef, align 4
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %126 = load <63 x i8>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %127 = load <3 x i1>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %128 = load <5 x i1>, ptr undef, align 4
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %129 = load <6 x i1>, ptr undef, align 4
@@ -4594,120 +4594,120 @@ define i32 @loads_partial_align4(i32 %arg) {
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %10 = load <7 x double>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %14 = load <5 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %14 = load <5 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %15 = load <6 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %16 = load <7 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %17 = load <9 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %17 = load <9 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %18 = load <10 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %19 = load <11 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %20 = load <12 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %21 = load <13 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %21 = load <13 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %22 = load <14 x i32>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %23 = load <15 x i32>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %27 = load <5 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %28 = load <6 x float>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %29 = load <7 x float>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %30 = load <9 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %31 = load <10 x float>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %32 = load <11 x float>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %33 = load <12 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %34 = load <13 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %35 = load <14 x float>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %36 = load <15 x float>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %39 = load <3 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %41 = load <5 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %43 = load <7 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %44 = load <9 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %45 = load <10 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %46 = load <11 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %45 = load <10 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %47 = load <12 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %48 = load <13 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %49 = load <14 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %50 = load <15 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %51 = load <17 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %52 = load <18 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %53 = load <19 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %50 = load <15 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %51 = load <17 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %55 = load <21 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %56 = load <22 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %57 = load <23 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %57 = load <23 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %59 = load <25 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %60 = load <26 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %61 = load <27 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %60 = load <26 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %61 = load <27 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %62 = load <28 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %63 = load <29 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %64 = load <30 x i16>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %65 = load <31 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %63 = load <29 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %65 = load <31 x i16>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %67 = load <2 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %68 = load <3 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %70 = load <5 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %71 = load <6 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %72 = load <7 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %74 = load <9 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %75 = load <10 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %78 = load <13 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %79 = load <14 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %80 = load <15 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %81 = load <17 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %82 = load <18 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %84 = load <20 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %85 = load <21 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %86 = load <22 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %87 = load <23 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %76 = load <11 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %80 = load <15 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %83 = load <19 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %84 = load <20 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %87 = load <23 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %88 = load <24 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %89 = load <25 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %90 = load <26 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %91 = load <27 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %92 = load <28 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %93 = load <29 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %94 = load <30 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %95 = load <31 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %96 = load <33 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %97 = load <34 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %98 = load <35 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %99 = load <36 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %100 = load <37 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %101 = load <38 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %102 = load <39 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %91 = load <27 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %93 = load <29 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %94 = load <30 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %95 = load <31 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %96 = load <33 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %97 = load <34 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %98 = load <35 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %102 = load <39 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %104 = load <41 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %105 = load <42 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %106 = load <43 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %107 = load <44 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %108 = load <45 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %109 = load <46 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %110 = load <47 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %106 = load <43 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %108 = load <45 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %109 = load <46 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %110 = load <47 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %112 = load <49 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %113 = load <50 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %114 = load <51 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %115 = load <52 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %116 = load <53 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %117 = load <54 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %118 = load <55 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %114 = load <51 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %115 = load <52 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %116 = load <53 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %117 = load <54 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %118 = load <55 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %119 = load <56 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %120 = load <57 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %121 = load <58 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %122 = load <59 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %123 = load <60 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %124 = load <61 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %125 = load <62 x i8>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %126 = load <63 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %120 = load <57 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %121 = load <58 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %124 = load <61 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %125 = load <62 x i8>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %126 = load <63 x i8>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 4
@@ -4734,37 +4734,37 @@ define i32 @loads_partial_align4(i32 %arg) {
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %150 = load <29 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %151 = load <30 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %152 = load <31 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %153 = load <33 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %154 = load <34 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %155 = load <35 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %156 = load <36 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %157 = load <37 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %158 = load <38 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %159 = load <39 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %160 = load <40 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %161 = load <41 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %162 = load <42 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %163 = load <43 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %164 = load <44 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %165 = load <45 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %166 = load <46 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %167 = load <47 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %168 = load <48 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %169 = load <49 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %170 = load <50 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %171 = load <51 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %172 = load <52 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %173 = load <53 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %174 = load <54 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %175 = load <55 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %176 = load <56 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %177 = load <57 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %178 = load <58 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %179 = load <59 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %180 = load <60 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %181 = load <61 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %182 = load <62 x i1>, ptr undef, align 4
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %183 = load <63 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %153 = load <33 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %154 = load <34 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %155 = load <35 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %156 = load <36 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %157 = load <37 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %158 = load <38 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %159 = load <39 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %160 = load <40 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %161 = load <41 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %162 = load <42 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %163 = load <43 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %164 = load <44 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %165 = load <45 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %166 = load <46 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %167 = load <47 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %168 = load <48 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %169 = load <49 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %170 = load <50 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %171 = load <51 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %172 = load <52 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %173 = load <53 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %174 = load <54 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %175 = load <55 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %176 = load <56 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %177 = load <57 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %178 = load <58 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %179 = load <59 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %180 = load <60 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %181 = load <61 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %182 = load <62 x i1>, ptr undef, align 4
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %183 = load <63 x i1>, ptr undef, align 4
; AVX512-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
load <1 x i64>, ptr undef, align 4
@@ -5308,94 +5308,94 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %36 = load <15 x float>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %39 = load <3 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %41 = load <5 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %42 = load <6 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %43 = load <7 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %43 = load <7 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %45 = load <10 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %46 = load <11 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %46 = load <11 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %47 = load <12 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %48 = load <13 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %48 = load <13 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %49 = load <14 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %50 = load <15 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %50 = load <15 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %51 = load <17 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %53 = load <19 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %55 = load <21 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %56 = load <22 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %57 = load <23 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %57 = load <23 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %61 = load <27 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %63 = load <29 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %64 = load <30 x i16>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %65 = load <31 x i16>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %65 = load <31 x i16>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %67 = load <2 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %68 = load <3 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 19 for instruction: %70 = load <5 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %71 = load <6 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %72 = load <7 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %72 = load <7 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 19 for instruction: %74 = load <9 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %75 = load <10 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %76 = load <11 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %76 = load <11 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %77 = load <12 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 24 for instruction: %78 = load <13 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %79 = load <14 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 27 for instruction: %80 = load <15 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %79 = load <14 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 25 for instruction: %80 = load <15 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %83 = load <19 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %84 = load <20 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %85 = load <21 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %86 = load <22 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %87 = load <23 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %86 = load <22 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %87 = load <23 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %88 = load <24 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %89 = load <25 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %90 = load <26 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %91 = load <27 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %90 = load <26 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %91 = load <27 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %92 = load <28 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 25 for instruction: %93 = load <29 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %94 = load <30 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 28 for instruction: %95 = load <31 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %94 = load <30 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 26 for instruction: %95 = load <31 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %96 = load <33 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %97 = load <34 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %98 = load <35 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %100 = load <37 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %101 = load <38 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 24 for instruction: %102 = load <39 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %102 = load <39 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 21 for instruction: %104 = load <41 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %105 = load <42 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 24 for instruction: %106 = load <43 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %106 = load <43 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %107 = load <44 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 26 for instruction: %108 = load <45 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %109 = load <46 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 29 for instruction: %110 = load <47 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %109 = load <46 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 27 for instruction: %110 = load <47 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %114 = load <51 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %116 = load <53 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %117 = load <54 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 25 for instruction: %118 = load <55 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %118 = load <55 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 22 for instruction: %120 = load <57 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %121 = load <58 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 25 for instruction: %122 = load <59 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 23 for instruction: %122 = load <59 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %123 = load <60 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 27 for instruction: %124 = load <61 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %125 = load <62 x i8>, ptr undef, align 1
-; SSE2-NEXT: Cost Model: Found an estimated cost of 30 for instruction: %126 = load <63 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %125 = load <62 x i8>, ptr undef, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 28 for instruction: %126 = load <63 x i8>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 1
; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 1
@@ -5468,131 +5468,131 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %10 = load <7 x double>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %14 = load <5 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %15 = load <6 x i32>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %16 = load <7 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %17 = load <9 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %18 = load <10 x i32>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %19 = load <11 x i32>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %20 = load <12 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %21 = load <13 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %22 = load <14 x i32>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %23 = load <15 x i32>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %27 = load <5 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %28 = load <6 x float>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %29 = load <7 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %30 = load <9 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %31 = load <10 x float>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %32 = load <11 x float>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %33 = load <12 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %34 = load <13 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %35 = load <14 x float>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %36 = load <15 x float>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %39 = load <3 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %41 = load <5 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %43 = load <7 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %45 = load <10 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %46 = load <11 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %46 = load <11 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %47 = load <12 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %48 = load <13 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %50 = load <15 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %48 = load <13 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %49 = load <14 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %50 = load <15 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %51 = load <17 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %53 = load <19 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %55 = load <21 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %56 = load <22 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %57 = load <23 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %57 = load <23 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %61 = load <27 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %63 = load <29 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %65 = load <31 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %64 = load <30 x i16>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %65 = load <31 x i16>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %67 = load <2 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %68 = load <3 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %68 = load <3 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %70 = load <5 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %71 = load <6 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %72 = load <7 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %74 = load <9 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %75 = load <10 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %76 = load <11 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %78 = load <13 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %79 = load <14 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %80 = load <15 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %80 = load <15 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %83 = load <19 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %84 = load <20 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %85 = load <21 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %86 = load <22 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %87 = load <23 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %85 = load <21 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %86 = load <22 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %87 = load <23 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %88 = load <24 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %89 = load <25 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %90 = load <26 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %91 = load <27 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %93 = load <29 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %94 = load <30 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %95 = load <31 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %89 = load <25 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %90 = load <26 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %91 = load <27 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %92 = load <28 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %93 = load <29 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %94 = load <30 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %95 = load <31 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %96 = load <33 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %97 = load <34 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %98 = load <35 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %98 = load <35 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %100 = load <37 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %101 = load <38 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %102 = load <39 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %102 = load <39 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %104 = load <41 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %105 = load <42 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %106 = load <43 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %107 = load <44 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %108 = load <45 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %109 = load <46 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %110 = load <47 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %106 = load <43 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %108 = load <45 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %109 = load <46 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %110 = load <47 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %114 = load <51 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %114 = load <51 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %116 = load <53 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %117 = load <54 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %118 = load <55 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %116 = load <53 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %118 = load <55 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %120 = load <57 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %121 = load <58 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %122 = load <59 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %124 = load <61 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %125 = load <62 x i8>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %126 = load <63 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %120 = load <57 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %123 = load <60 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %124 = load <61 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %125 = load <62 x i8>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %126 = load <63 x i8>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %130 = load <7 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %131 = load <9 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %132 = load <10 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %133 = load <11 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %134 = load <12 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %135 = load <13 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %136 = load <14 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %137 = load <15 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %131 = load <9 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %132 = load <10 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %133 = load <11 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %134 = load <12 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %135 = load <13 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %136 = load <14 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %137 = load <15 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %138 = load <17 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %139 = load <18 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %140 = load <19 x i1>, ptr undef, align 1
@@ -5601,13 +5601,13 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %143 = load <22 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %144 = load <23 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %145 = load <24 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %146 = load <25 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %147 = load <26 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %148 = load <27 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %149 = load <28 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %150 = load <29 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %151 = load <30 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %152 = load <31 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %146 = load <25 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %147 = load <26 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %148 = load <27 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %149 = load <28 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %150 = load <29 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %151 = load <30 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %152 = load <31 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %153 = load <33 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %154 = load <34 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %155 = load <35 x i1>, ptr undef, align 1
@@ -5616,13 +5616,13 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %158 = load <38 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %159 = load <39 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %160 = load <40 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %161 = load <41 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %162 = load <42 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %163 = load <43 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %164 = load <44 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %165 = load <45 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %166 = load <46 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %167 = load <47 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %161 = load <41 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %162 = load <42 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %163 = load <43 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %164 = load <44 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %165 = load <45 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %166 = load <46 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %167 = load <47 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %168 = load <48 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %169 = load <49 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %170 = load <50 x i1>, ptr undef, align 1
@@ -5632,13 +5632,13 @@ define i32 @loads_partial_align1(i32 %arg) {
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %174 = load <54 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %175 = load <55 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %176 = load <56 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %177 = load <57 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %178 = load <58 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %179 = load <59 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %180 = load <60 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %181 = load <61 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %182 = load <62 x i1>, ptr undef, align 1
-; SSE41-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %183 = load <63 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %177 = load <57 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %178 = load <58 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %179 = load <59 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %180 = load <60 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %181 = load <61 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %182 = load <62 x i1>, ptr undef, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %183 = load <63 x i1>, ptr undef, align 1
; SSE41-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
; AVX-LABEL: 'loads_partial_align1'
@@ -5654,146 +5654,146 @@ define i32 @loads_partial_align1(i32 %arg) {
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %10 = load <7 x double>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %14 = load <5 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %14 = load <5 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %15 = load <6 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %16 = load <7 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %17 = load <9 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %18 = load <10 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %19 = load <11 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %20 = load <12 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %21 = load <13 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %21 = load <13 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %22 = load <14 x i32>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %23 = load <15 x i32>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %27 = load <5 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %28 = load <6 x float>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %29 = load <7 x float>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %30 = load <9 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %31 = load <10 x float>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %32 = load <11 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %33 = load <12 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %34 = load <13 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %35 = load <14 x float>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %36 = load <15 x float>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %39 = load <3 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %41 = load <5 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %43 = load <7 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %44 = load <9 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %45 = load <10 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %46 = load <11 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %45 = load <10 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %47 = load <12 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %48 = load <13 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %49 = load <14 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %50 = load <15 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %50 = load <15 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %51 = load <17 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %52 = load <18 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %53 = load <19 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %53 = load <19 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %54 = load <20 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %55 = load <21 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %57 = load <23 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %55 = load <21 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %56 = load <22 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %57 = load <23 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %58 = load <24 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %59 = load <25 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %60 = load <26 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %61 = load <27 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %59 = load <25 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %60 = load <26 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %61 = load <27 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %62 = load <28 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %63 = load <29 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %65 = load <31 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %63 = load <29 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %64 = load <30 x i16>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %65 = load <31 x i16>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %67 = load <2 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %68 = load <3 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %68 = load <3 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %70 = load <5 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %71 = load <6 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %72 = load <7 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %74 = load <9 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %75 = load <10 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %76 = load <11 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %78 = load <13 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %79 = load <14 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %80 = load <15 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %81 = load <17 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %82 = load <18 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %83 = load <19 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %84 = load <20 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %85 = load <21 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %86 = load <22 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %87 = load <23 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %80 = load <15 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %84 = load <20 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %87 = load <23 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %88 = load <24 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %89 = load <25 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %90 = load <26 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %91 = load <27 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %92 = load <28 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %93 = load <29 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %94 = load <30 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %95 = load <31 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %91 = load <27 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %93 = load <29 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %94 = load <30 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %95 = load <31 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %96 = load <33 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %97 = load <34 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %98 = load <35 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %98 = load <35 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %99 = load <36 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %100 = load <37 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %101 = load <38 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %102 = load <39 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %100 = load <37 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %101 = load <38 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %102 = load <39 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %103 = load <40 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %104 = load <41 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %105 = load <42 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %106 = load <43 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %108 = load <45 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %109 = load <46 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %110 = load <47 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %104 = load <41 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %105 = load <42 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %106 = load <43 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %107 = load <44 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %108 = load <45 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %109 = load <46 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %110 = load <47 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %111 = load <48 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %112 = load <49 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %113 = load <50 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %114 = load <51 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %115 = load <52 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %116 = load <53 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %117 = load <54 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %118 = load <55 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %112 = load <49 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %113 = load <50 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %114 = load <51 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %115 = load <52 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %116 = load <53 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %117 = load <54 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %118 = load <55 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %119 = load <56 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %120 = load <57 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %121 = load <58 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %122 = load <59 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %124 = load <61 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %125 = load <62 x i8>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %126 = load <63 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %120 = load <57 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %121 = load <58 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %122 = load <59 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %123 = load <60 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %124 = load <61 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %125 = load <62 x i8>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %126 = load <63 x i8>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %130 = load <7 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %131 = load <9 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %132 = load <10 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %133 = load <11 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %134 = load <12 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %135 = load <13 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %136 = load <14 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %137 = load <15 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %138 = load <17 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %139 = load <18 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %140 = load <19 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %141 = load <20 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %142 = load <21 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %143 = load <22 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %144 = load <23 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %145 = load <24 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %146 = load <25 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %147 = load <26 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %148 = load <27 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %149 = load <28 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %150 = load <29 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %151 = load <30 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %152 = load <31 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %131 = load <9 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %132 = load <10 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %133 = load <11 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %134 = load <12 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %135 = load <13 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %136 = load <14 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %137 = load <15 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %138 = load <17 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %139 = load <18 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %140 = load <19 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %141 = load <20 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %142 = load <21 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %143 = load <22 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %144 = load <23 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %145 = load <24 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %146 = load <25 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %147 = load <26 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %148 = load <27 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %149 = load <28 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %150 = load <29 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %151 = load <30 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %152 = load <31 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %153 = load <33 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %154 = load <34 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %155 = load <35 x i1>, ptr undef, align 1
@@ -5802,29 +5802,29 @@ define i32 @loads_partial_align1(i32 %arg) {
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %158 = load <38 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %159 = load <39 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %160 = load <40 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %161 = load <41 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %162 = load <42 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %163 = load <43 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %164 = load <44 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %165 = load <45 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %166 = load <46 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %167 = load <47 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %161 = load <41 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %162 = load <42 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %163 = load <43 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %164 = load <44 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %165 = load <45 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %166 = load <46 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %167 = load <47 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %168 = load <48 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %169 = load <49 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %170 = load <50 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %171 = load <51 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %172 = load <52 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %173 = load <53 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %174 = load <54 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %175 = load <55 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %176 = load <56 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %177 = load <57 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %178 = load <58 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %179 = load <59 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %180 = load <60 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %181 = load <61 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %182 = load <62 x i1>, ptr undef, align 1
-; AVX-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %183 = load <63 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %169 = load <49 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %170 = load <50 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %171 = load <51 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %172 = load <52 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %173 = load <53 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %174 = load <54 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %175 = load <55 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %176 = load <56 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %177 = load <57 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %178 = load <58 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %179 = load <59 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %180 = load <60 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %181 = load <61 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %182 = load <62 x i1>, ptr undef, align 1
+; AVX-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %183 = load <63 x i1>, ptr undef, align 1
; AVX-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
; AVX512-LABEL: 'loads_partial_align1'
@@ -5840,177 +5840,177 @@ define i32 @loads_partial_align1(i32 %arg) {
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %10 = load <7 x double>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = load <1 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = load <2 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %13 = load <3 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %14 = load <5 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = load <3 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %14 = load <5 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %15 = load <6 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %16 = load <7 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %17 = load <9 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %16 = load <7 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %17 = load <9 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %18 = load <10 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %19 = load <11 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %19 = load <11 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %20 = load <12 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %21 = load <13 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %21 = load <13 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %22 = load <14 x i32>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %23 = load <15 x i32>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %23 = load <15 x i32>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %24 = load <1 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %25 = load <2 x float>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %26 = load <3 x float>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = load <3 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %27 = load <5 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %28 = load <6 x float>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %29 = load <7 x float>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %29 = load <7 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %30 = load <9 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %31 = load <10 x float>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %32 = load <11 x float>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = load <11 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %33 = load <12 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %34 = load <13 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %35 = load <14 x float>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %36 = load <15 x float>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %36 = load <15 x float>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %37 = load <1 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %38 = load <2 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %39 = load <3 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %39 = load <3 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %40 = load <4 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %41 = load <5 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %42 = load <6 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %43 = load <7 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %44 = load <9 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %45 = load <10 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %46 = load <11 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %41 = load <5 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %42 = load <6 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %43 = load <7 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %44 = load <9 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %45 = load <10 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %46 = load <11 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %47 = load <12 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %48 = load <13 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %49 = load <14 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %50 = load <15 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %51 = load <17 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %52 = load <18 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %53 = load <19 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %48 = load <13 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %49 = load <14 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %50 = load <15 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %51 = load <17 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %52 = load <18 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %53 = load <19 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %54 = load <20 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %55 = load <21 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %56 = load <22 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %57 = load <23 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %55 = load <21 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %56 = load <22 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %57 = load <23 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %58 = load <24 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %59 = load <25 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %60 = load <26 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %61 = load <27 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %59 = load <25 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %60 = load <26 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %61 = load <27 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %62 = load <28 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %63 = load <29 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %64 = load <30 x i16>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %65 = load <31 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %63 = load <29 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %64 = load <30 x i16>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %65 = load <31 x i16>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %66 = load <1 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %67 = load <2 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %68 = load <3 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %68 = load <3 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %69 = load <4 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %70 = load <5 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %71 = load <6 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %72 = load <7 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %70 = load <5 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %71 = load <6 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %72 = load <7 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %73 = load <8 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %74 = load <9 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %75 = load <10 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %76 = load <11 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %77 = load <12 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %78 = load <13 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %79 = load <14 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %80 = load <15 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %81 = load <17 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %82 = load <18 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %83 = load <19 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %84 = load <20 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %85 = load <21 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %86 = load <22 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %87 = load <23 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %74 = load <9 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %75 = load <10 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %76 = load <11 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %77 = load <12 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %78 = load <13 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %79 = load <14 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %80 = load <15 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %81 = load <17 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %82 = load <18 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %83 = load <19 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %84 = load <20 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %85 = load <21 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %86 = load <22 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %87 = load <23 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %88 = load <24 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %89 = load <25 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %90 = load <26 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %91 = load <27 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %92 = load <28 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %93 = load <29 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %94 = load <30 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %95 = load <31 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %96 = load <33 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %97 = load <34 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %98 = load <35 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %99 = load <36 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %100 = load <37 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %101 = load <38 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %102 = load <39 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %89 = load <25 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %90 = load <26 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %91 = load <27 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %92 = load <28 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %93 = load <29 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %94 = load <30 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %95 = load <31 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %96 = load <33 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %97 = load <34 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %98 = load <35 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %99 = load <36 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %100 = load <37 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %101 = load <38 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %102 = load <39 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %103 = load <40 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %104 = load <41 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %105 = load <42 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %106 = load <43 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %107 = load <44 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %108 = load <45 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %109 = load <46 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %110 = load <47 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %104 = load <41 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %105 = load <42 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %106 = load <43 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %107 = load <44 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %108 = load <45 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %109 = load <46 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %110 = load <47 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %111 = load <48 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %112 = load <49 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %113 = load <50 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %114 = load <51 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %115 = load <52 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %116 = load <53 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %117 = load <54 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %118 = load <55 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %112 = load <49 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %113 = load <50 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %114 = load <51 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %115 = load <52 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %116 = load <53 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %117 = load <54 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %118 = load <55 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %119 = load <56 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %120 = load <57 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %121 = load <58 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %122 = load <59 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %123 = load <60 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %124 = load <61 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %125 = load <62 x i8>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 13 for instruction: %126 = load <63 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %120 = load <57 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %121 = load <58 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %122 = load <59 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %123 = load <60 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %124 = load <61 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %125 = load <62 x i8>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %126 = load <63 x i8>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %127 = load <3 x i1>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %128 = load <5 x i1>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %129 = load <6 x i1>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %130 = load <7 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %131 = load <9 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %132 = load <10 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %133 = load <11 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %134 = load <12 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %135 = load <13 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %136 = load <14 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %137 = load <15 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %138 = load <17 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %139 = load <18 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %140 = load <19 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %141 = load <20 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %142 = load <21 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %143 = load <22 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %144 = load <23 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %145 = load <24 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %146 = load <25 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %147 = load <26 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %148 = load <27 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %149 = load <28 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %150 = load <29 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %151 = load <30 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %152 = load <31 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %153 = load <33 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %154 = load <34 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %155 = load <35 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %156 = load <36 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %157 = load <37 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %158 = load <38 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %159 = load <39 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %160 = load <40 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %161 = load <41 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %162 = load <42 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %163 = load <43 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %164 = load <44 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %165 = load <45 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %166 = load <46 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %167 = load <47 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %168 = load <48 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %169 = load <49 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %170 = load <50 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %171 = load <51 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %172 = load <52 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %173 = load <53 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %174 = load <54 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %175 = load <55 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %176 = load <56 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %177 = load <57 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %178 = load <58 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %179 = load <59 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %180 = load <60 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %181 = load <61 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %182 = load <62 x i1>, ptr undef, align 1
-; AVX512-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %183 = load <63 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %131 = load <9 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %132 = load <10 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %133 = load <11 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %134 = load <12 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %135 = load <13 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %136 = load <14 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %137 = load <15 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %138 = load <17 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %139 = load <18 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %140 = load <19 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %141 = load <20 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %142 = load <21 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %143 = load <22 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %144 = load <23 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %145 = load <24 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %146 = load <25 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %147 = load <26 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %148 = load <27 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %149 = load <28 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %150 = load <29 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %151 = load <30 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %152 = load <31 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %153 = load <33 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %154 = load <34 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %155 = load <35 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %156 = load <36 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %157 = load <37 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %158 = load <38 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %159 = load <39 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %160 = load <40 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %161 = load <41 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %162 = load <42 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %163 = load <43 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %164 = load <44 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %165 = load <45 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %166 = load <46 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %167 = load <47 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %168 = load <48 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %169 = load <49 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %170 = load <50 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %171 = load <51 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %172 = load <52 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %173 = load <53 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %174 = load <54 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %175 = load <55 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %176 = load <56 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %177 = load <57 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %178 = load <58 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %179 = load <59 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %180 = load <60 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %181 = load <61 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %182 = load <62 x i1>, ptr undef, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %183 = load <63 x i1>, ptr undef, align 1
; AVX512-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
load <1 x i64>, ptr undef, align 1
diff --git a/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll b/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
new file mode 100644
index 00000000000000..e6a79f13dfa68a
--- /dev/null
+++ b/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
@@ -0,0 +1,68 @@
+; NOTE: Assertions have been autogenerated by utils/update_analyze_test_checks.py
+; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+sse2 | FileCheck %s --check-prefixes=SSE2
+; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+sse4.1 | FileCheck %s --check-prefixes=SSE41
+; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+avx2 | FileCheck %s --check-prefixes=AVX2
+
+; Non-power-of-2 sub-16B vector loads are materialized by a base load plus a
+; folded vpinsr*(mem) per remaining in-lane chunk (1B->pinsrb, 2B->pinsrw,
+; 4B->pinsrd), i.e. one instruction per pow2 chunk. The cost must reflect that
+; (popcount(VF) loads for in-lane types, plus real vinserti128 stitches), not
+; double-count the folded insert. The fold is ISA-gated: the byte (pinsrb) and
+; dword (pinsrd) folds need SSE4.1, the word (pinsrw) fold only needs SSE2, so
+; on SSE2 <3 x i8> stays high while <3 x i16> already gets the relief. The same
+; applies to FP (the 4-byte float remainder folds via INSERTPS/SSE4.1), while
+; 8-byte chunks (<3 x i64>/<3 x double>) are outside the fold set and unchanged.
+; Power-of-2 loads (e.g. <2 x i8>) are left as-is.
+
+define void @npot_loads(ptr %p) {
+; SSE2-LABEL: 'npot_loads'
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %v3i8 = load <3 x i8>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %v7i8 = load <7 x i8>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i16 = load <3 x i16>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %v3i32 = load <3 x i32>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %v7i32 = load <7 x i32>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i64 = load <3 x i64>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %v3f32 = load <3 x float>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
+; SSE2-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
+;
+; SSE41-LABEL: 'npot_loads'
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i8 = load <3 x i8>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7i8 = load <7 x i8>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i16 = load <3 x i16>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i32 = load <3 x i32>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7i32 = load <7 x i32>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i64 = load <3 x i64>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f32 = load <3 x float>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
+; SSE41-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
+;
+; AVX2-LABEL: 'npot_loads'
+; AVX2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i8 = load <3 x i8>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7i8 = load <7 x i8>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i16 = load <3 x i16>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i32 = load <3 x i32>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7i32 = load <7 x i32>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i64 = load <3 x i64>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f32 = load <3 x float>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
+; AVX2-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
+;
+ %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+ %v3i8 = load <3 x i8>, ptr %p, align 1
+ %v7i8 = load <7 x i8>, ptr %p, align 1
+ %v3i16 = load <3 x i16>, ptr %p, align 1
+ %v3i32 = load <3 x i32>, ptr %p, align 1
+ %v7i32 = load <7 x i32>, ptr %p, align 1
+ %v3i64 = load <3 x i64>, ptr %p, align 1
+ %v3f32 = load <3 x float>, ptr %p, align 1
+ %v7f32 = load <7 x float>, ptr %p, align 1
+ %v3f64 = load <3 x double>, ptr %p, align 1
+ ret void
+}
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-3.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-3.ll
index 15baa2a627f234..156bd432a34477 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-3.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-3.ll
@@ -30,7 +30,7 @@ define void @test() {
;
; AVX2-LABEL: 'test'
; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i16, ptr %in0, align 2
-; AVX2: Cost of 8 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX2: Cost of 7 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX2: ir<%v0> = load from index 0
; AVX2: ir<%v1> = load from index 1
; AVX2: ir<%v2> = load from index 2
@@ -53,7 +53,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i16, ptr %in0, align 2
-; AVX512DQ: Cost of 8 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX512DQ: Cost of 7 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-5.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-5.ll
index 66714ab44b62af..a7b6bf588af099 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-5.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-5.ll
@@ -38,7 +38,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i16, ptr %in0, align 2
-; AVX512DQ: Cost of 25 for VF 2: INTERLEAVE-GROUP with factor 5, ir<%in0>
+; AVX512DQ: Cost of 24 for VF 2: INTERLEAVE-GROUP with factor 5, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-7.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-7.ll
index dcbf02f16f68d2..26e82797634251 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-7.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i16-stride-7.ll
@@ -38,7 +38,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i16, ptr %in0, align 2
-; AVX512DQ: Cost of 34 for VF 2: INTERLEAVE-GROUP with factor 7, ir<%in0>
+; AVX512DQ: Cost of 33 for VF 2: INTERLEAVE-GROUP with factor 7, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-3.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-3.ll
index e3224b8938f7c8..f385a46789ecc2 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-3.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-3.ll
@@ -30,11 +30,11 @@ define void @test() {
;
; AVX2-LABEL: 'test'
; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX2: Cost of 7 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX2: Cost of 5 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX2: ir<%v0> = load from index 0
; AVX2: ir<%v1> = load from index 1
; AVX2: ir<%v2> = load from index 2
-; AVX2: Cost of 6 for VF 4: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX2: Cost of 5 for VF 4: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX2: ir<%v0> = load from index 0
; AVX2: ir<%v1> = load from index 1
; AVX2: ir<%v2> = load from index 2
@@ -53,11 +53,11 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX512DQ: Cost of 7 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX512DQ: Cost of 5 for VF 2: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
-; AVX512DQ: Cost of 6 for VF 4: INTERLEAVE-GROUP with factor 3, ir<%in0>
+; AVX512DQ: Cost of 5 for VF 4: INTERLEAVE-GROUP with factor 3, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-5.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-5.ll
index 3d887cb7958a91..490ac46b0b4ac9 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-5.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-5.ll
@@ -38,13 +38,13 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX512DQ: Cost of 24 for VF 2: INTERLEAVE-GROUP with factor 5, ir<%in0>
+; AVX512DQ: Cost of 22 for VF 2: INTERLEAVE-GROUP with factor 5, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
; AVX512DQ: ir<%v3> = load from index 3
; AVX512DQ: ir<%v4> = load from index 4
-; AVX512DQ: Cost of 45 for VF 4: INTERLEAVE-GROUP with factor 5, ir<%in0>
+; AVX512DQ: Cost of 44 for VF 4: INTERLEAVE-GROUP with factor 5, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-6.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-6.ll
index a2f22da5d7a4da..08c5f31fbd5d03 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-6.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-6.ll
@@ -30,7 +30,7 @@ define void @test() {
;
; AVX2-LABEL: 'test'
; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX2: Cost of 9 for VF 2: INTERLEAVE-GROUP with factor 6, ir<%in0>
+; AVX2: Cost of 8 for VF 2: INTERLEAVE-GROUP with factor 6, ir<%in0>
; AVX2: ir<%v0> = load from index 0
; AVX2: ir<%v1> = load from index 1
; AVX2: ir<%v2> = load from index 2
@@ -68,7 +68,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX512DQ: Cost of 9 for VF 2: INTERLEAVE-GROUP with factor 6, ir<%in0>
+; AVX512DQ: Cost of 8 for VF 2: INTERLEAVE-GROUP with factor 6, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-7.ll b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-7.ll
index d9d66e5b895986..afbe3c05bfb9e4 100644
--- a/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-7.ll
+++ b/llvm/test/Transforms/LoopVectorize/X86/CostModel/interleaved-load-i8-stride-7.ll
@@ -38,7 +38,7 @@ define void @test() {
;
; AVX512DQ-LABEL: 'test'
; AVX512DQ: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, ptr %in0, align 1
-; AVX512DQ: Cost of 34 for VF 2: INTERLEAVE-GROUP with factor 7, ir<%in0>
+; AVX512DQ: Cost of 31 for VF 2: INTERLEAVE-GROUP with factor 7, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
@@ -46,7 +46,7 @@ define void @test() {
; AVX512DQ: ir<%v4> = load from index 4
; AVX512DQ: ir<%v5> = load from index 5
; AVX512DQ: ir<%v6> = load from index 6
-; AVX512DQ: Cost of 62 for VF 4: INTERLEAVE-GROUP with factor 7, ir<%in0>
+; AVX512DQ: Cost of 61 for VF 4: INTERLEAVE-GROUP with factor 7, ir<%in0>
; AVX512DQ: ir<%v0> = load from index 0
; AVX512DQ: ir<%v1> = load from index 1
; AVX512DQ: ir<%v2> = load from index 2
diff --git a/llvm/test/Transforms/SLPVectorizer/X86/nonpow2-shared-weight-reduction.ll b/llvm/test/Transforms/SLPVectorizer/X86/nonpow2-shared-weight-reduction.ll
index 846c135092ccca..a4bc9b6687dae4 100644
--- a/llvm/test/Transforms/SLPVectorizer/X86/nonpow2-shared-weight-reduction.ll
+++ b/llvm/test/Transforms/SLPVectorizer/X86/nonpow2-shared-weight-reduction.ll
@@ -18,46 +18,33 @@ define void @ship3(ptr %weights, ptr %src, i64 %bpp, ptr %dst, i32 %n) {
; NPOT-NEXT: br i1 [[POS]], label [[LOOP:%.*]], label [[EXIT:%.*]]
; NPOT: loop:
; NPOT-NEXT: [[IV:%.*]] = phi i64 [ 0, [[TMP0:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
-; NPOT-NEXT: [[ACCB:%.*]] = phi i32 [ 0, [[TMP0]] ], [ [[ADDB:%.*]], [[LOOP]] ]
-; NPOT-NEXT: [[ACCG:%.*]] = phi i32 [ 0, [[TMP0]] ], [ [[ADDG:%.*]], [[LOOP]] ]
-; NPOT-NEXT: [[ACCR:%.*]] = phi i32 [ 0, [[TMP0]] ], [ [[ADDR:%.*]], [[LOOP]] ]
+; NPOT-NEXT: [[TMP1:%.*]] = phi <3 x i32> [ zeroinitializer, [[TMP0]] ], [ [[TMP7:%.*]], [[LOOP]] ]
; NPOT-NEXT: [[WGEP:%.*]] = getelementptr inbounds i32, ptr [[WEIGHTS:%.*]], i64 [[IV]]
; NPOT-NEXT: [[W:%.*]] = load i32, ptr [[WGEP]], align 4
; NPOT-NEXT: [[OFF:%.*]] = mul nsw i64 [[IV]], [[BPP:%.*]]
; NPOT-NEXT: [[P0:%.*]] = getelementptr inbounds i8, ptr [[SRC:%.*]], i64 [[OFF]]
-; NPOT-NEXT: [[P1:%.*]] = getelementptr inbounds i8, ptr [[P0]], i64 1
-; NPOT-NEXT: [[P2:%.*]] = getelementptr inbounds i8, ptr [[P0]], i64 2
-; NPOT-NEXT: [[B:%.*]] = load i8, ptr [[P0]], align 1
-; NPOT-NEXT: [[G:%.*]] = load i8, ptr [[P1]], align 1
-; NPOT-NEXT: [[R:%.*]] = load i8, ptr [[P2]], align 1
-; NPOT-NEXT: [[ZB:%.*]] = zext i8 [[B]] to i32
-; NPOT-NEXT: [[ZG:%.*]] = zext i8 [[G]] to i32
-; NPOT-NEXT: [[ZR:%.*]] = zext i8 [[R]] to i32
-; NPOT-NEXT: [[MB:%.*]] = mul nsw i32 [[W]], [[ZB]]
-; NPOT-NEXT: [[MG:%.*]] = mul nsw i32 [[W]], [[ZG]]
-; NPOT-NEXT: [[MR:%.*]] = mul nsw i32 [[W]], [[ZR]]
-; NPOT-NEXT: [[ADDB]] = add nsw i32 [[MB]], [[ACCB]]
-; NPOT-NEXT: [[ADDG]] = add nsw i32 [[MG]], [[ACCG]]
-; NPOT-NEXT: [[ADDR]] = add nsw i32 [[MR]], [[ACCR]]
+; NPOT-NEXT: [[TMP2:%.*]] = load <3 x i8>, ptr [[P0]], align 1
+; NPOT-NEXT: [[TMP3:%.*]] = zext <3 x i8> [[TMP2]] to <3 x i32>
+; NPOT-NEXT: [[TMP4:%.*]] = insertelement <3 x i32> poison, i32 [[W]], i64 0
+; NPOT-NEXT: [[TMP5:%.*]] = shufflevector <3 x i32> [[TMP4]], <3 x i32> poison, <3 x i32> zeroinitializer
+; NPOT-NEXT: [[TMP6:%.*]] = mul nsw <3 x i32> [[TMP5]], [[TMP3]]
+; NPOT-NEXT: [[TMP7]] = add nsw <3 x i32> [[TMP6]], [[TMP1]]
; NPOT-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1
; NPOT-NEXT: [[DONE:%.*]] = icmp eq i64 [[IV_NEXT]], [[N_EXT]]
; NPOT-NEXT: br i1 [[DONE]], label [[EXIT_LOOP:%.*]], label [[LOOP]]
; NPOT: exit.loop:
-; NPOT-NEXT: [[SB:%.*]] = lshr i32 [[ADDB]], 16
-; NPOT-NEXT: [[SG:%.*]] = lshr i32 [[ADDG]], 16
-; NPOT-NEXT: [[SR:%.*]] = lshr i32 [[ADDR]], 16
-; NPOT-NEXT: [[TB:%.*]] = trunc i32 [[SB]] to i8
-; NPOT-NEXT: [[TG:%.*]] = trunc i32 [[SG]] to i8
-; NPOT-NEXT: [[TR:%.*]] = trunc i32 [[SR]] to i8
+; NPOT-NEXT: [[TMP8:%.*]] = lshr <3 x i32> [[TMP7]], splat (i32 16)
+; NPOT-NEXT: [[TMP9:%.*]] = trunc <3 x i32> [[TMP8]] to <3 x i8>
; NPOT-NEXT: br label [[EXIT]]
; NPOT: exit:
-; NPOT-NEXT: [[TMP11:%.*]] = phi i8 [ 0, [[TMP0]] ], [ [[TB]], [[EXIT_LOOP]] ]
-; NPOT-NEXT: [[TMP12:%.*]] = phi i8 [ 0, [[TMP0]] ], [ [[TG]], [[EXIT_LOOP]] ]
-; NPOT-NEXT: [[TMP13:%.*]] = phi i8 [ 0, [[TMP0]] ], [ [[TR]], [[EXIT_LOOP]] ]
+; NPOT-NEXT: [[TMP10:%.*]] = phi <3 x i8> [ zeroinitializer, [[TMP0]] ], [ [[TMP9]], [[EXIT_LOOP]] ]
; NPOT-NEXT: [[D1:%.*]] = getelementptr inbounds i8, ptr [[DST:%.*]], i64 1
; NPOT-NEXT: [[D2:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 2
+; NPOT-NEXT: [[TMP11:%.*]] = extractelement <3 x i8> [[TMP10]], i64 0
; NPOT-NEXT: store i8 [[TMP11]], ptr [[DST]], align 1
+; NPOT-NEXT: [[TMP12:%.*]] = extractelement <3 x i8> [[TMP10]], i64 1
; NPOT-NEXT: store i8 [[TMP12]], ptr [[D1]], align 1
+; NPOT-NEXT: [[TMP13:%.*]] = extractelement <3 x i8> [[TMP10]], i64 2
; NPOT-NEXT: store i8 [[TMP13]], ptr [[D2]], align 1
; NPOT-NEXT: ret void
;
>From 756d819384fba6184014c27e8f687465e0dc5616 Mon Sep 17 00:00:00 2001
From: mbhade <mbhade at amd.com>
Date: Tue, 1 Sep 2026 12:48:53 +0530
Subject: [PATCH 2/3] [X86] NFC: Format folded insert cost logic
Apply clang-format to the folded load-insert cost calculation.
Co-authored-by: Cursor <cursoragent at cursor.com>
---
llvm/lib/Target/X86/X86TargetTransformInfo.cpp | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index e30cceec912cda..54510df4abfe4a 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -5906,11 +5906,10 @@ InstructionCost X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
// below must be skipped - otherwise the folded insert is double-counted.
// Stores (the symmetric PEXTR*(mem) fold) are left unchanged here.
bool Is0thSubVec = (NumEltDone() % LT.second.getVectorNumElements()) == 0;
- bool FoldedInLaneInsert =
- IsLoad && !Is0thSubVec &&
- ((CurrOpSizeBytes == 1 && ST->hasSSE41()) ||
- (CurrOpSizeBytes == 2 && ST->hasSSE2()) ||
- (CurrOpSizeBytes == 4 && ST->hasSSE41()));
+ bool FoldedInLaneInsert = IsLoad && !Is0thSubVec &&
+ ((CurrOpSizeBytes == 1 && ST->hasSSE41()) ||
+ (CurrOpSizeBytes == 2 && ST->hasSSE2()) ||
+ (CurrOpSizeBytes == 4 && ST->hasSSE41()));
if (CurrOpSizeBytes == 32 && ST->isUnalignedMem32Slow())
Cost += 2;
else if (CurrOpSizeBytes < 4 && !FoldedInLaneInsert)
>From afa677e677b7cc8449982df4fb4fc30287528c9a Mon Sep 17 00:00:00 2001
From: mbhade <mbhade at amd.com>
Date: Wed, 9 Sep 2026 10:54:07 +0530
Subject: [PATCH 3/3] [X86] Add AVX512 coverage to folded-load cost test
Extend nonpow2-load-fold.ll with an -mattr=+avx512f,+avx512vl RUN line so
the corrected non-power-of-2 folded-insert load costs are checked at the
AVX512 ISA level as well.
Co-authored-by: Cursor <cursoragent at cursor.com>
---
.../Analysis/CostModel/X86/nonpow2-load-fold.ll | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll b/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
index e6a79f13dfa68a..abd529be18b7b0 100644
--- a/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
+++ b/llvm/test/Analysis/CostModel/X86/nonpow2-load-fold.ll
@@ -2,6 +2,7 @@
; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+sse2 | FileCheck %s --check-prefixes=SSE2
; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+sse4.1 | FileCheck %s --check-prefixes=SSE41
; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+avx2 | FileCheck %s --check-prefixes=AVX2
+; RUN: opt < %s -passes="print<cost-model>" 2>&1 -disable-output -mtriple=x86_64-unknown-linux-gnu -mattr=+avx512f,+avx512vl | FileCheck %s --check-prefixes=AVX512
; Non-power-of-2 sub-16B vector loads are materialized by a base load plus a
; folded vpinsr*(mem) per remaining in-lane chunk (1B->pinsrb, 2B->pinsrw,
@@ -53,6 +54,19 @@ define void @npot_loads(ptr %p) {
; AVX2-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
; AVX2-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
; AVX2-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
+;
+; AVX512-LABEL: 'npot_loads'
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %pow2_v2i8 = load <2 x i8>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i8 = load <3 x i8>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v7i8 = load <7 x i8>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i16 = load <3 x i16>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3i32 = load <3 x i32>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7i32 = load <7 x i32>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3i64 = load <3 x i64>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %v3f32 = load <3 x float>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %v7f32 = load <7 x float>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %v3f64 = load <3 x double>, ptr %p, align 1
+; AVX512-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
%pow2_v2i8 = load <2 x i8>, ptr %p, align 1
%v3i8 = load <3 x i8>, ptr %p, align 1
More information about the llvm-commits
mailing list