[llvm] [SLP] Add store-to-load forwarding conflict cost for widened store chains (PR #199606)
Alexey Bataev via llvm-commits
llvm-commits at lists.llvm.org
Sun Sep 13 12:01:41 PDT 2026
================
@@ -17973,6 +17991,24 @@ BoUpSLP::getEntryCost(const TreeEntry *E, ArrayRef<Value *> VectorizedVals,
case TreeEntry::NeedToGather:
llvm_unreachable("Unexpected vectorization state.");
}
+ // Widening this load can break store-to-load forwarding for a nearby
+ // loop-carried store that stays scalar (or narrower) in this tree.
+ // Mirror the store-side model: add the target's modeled STLF penalty
+ // instead of rejecting the tree, and only under throughput/latency cost
+ // kinds. Use the number of distinct loaded scalars (not the reuse-
+ // inflated vector factor), scaled by the interleave factor, for both
+ // the hazard's distance check and the penalty type. Factor 0 means
+ // non-interleaved.
+ unsigned STLFLoadVF =
+ E->Scalars.size() * std::max(1u, E->getInterleaveFactor());
+ if (EnableSLPStoreLoadForwardCheck && E->State == TreeEntry::Vectorize &&
----------------
alexey-bataev wrote:
Add StridedVectorize, ScatterVectorize, ExpandVectorize, CompressVectorize, BlendedLoadVectorize
https://github.com/llvm/llvm-project/pull/199606
More information about the llvm-commits
mailing list