[llvm] [SLP] Add store-to-load forwarding conflict cost for widened store chains (PR #199606)

Alexey Bataev via llvm-commits llvm-commits at lists.llvm.org
Sun Sep 13 12:01:41 PDT 2026


================
@@ -17973,6 +17991,24 @@ BoUpSLP::getEntryCost(const TreeEntry *E, ArrayRef<Value *> VectorizedVals,
       case TreeEntry::NeedToGather:
         llvm_unreachable("Unexpected vectorization state.");
       }
+      // Widening this load can break store-to-load forwarding for a nearby
+      // loop-carried store that stays scalar (or narrower) in this tree.
+      // Mirror the store-side model: add the target's modeled STLF penalty
+      // instead of rejecting the tree, and only under throughput/latency cost
+      // kinds. Use the number of distinct loaded scalars (not the reuse-
+      // inflated vector factor), scaled by the interleave factor, for both
+      // the hazard's distance check and the penalty type. Factor 0 means
+      // non-interleaved.
+      unsigned STLFLoadVF =
+          E->Scalars.size() * std::max(1u, E->getInterleaveFactor());
+      if (EnableSLPStoreLoadForwardCheck && E->State == TreeEntry::Vectorize &&
----------------
alexey-bataev wrote:

Add StridedVectorize, ScatterVectorize, ExpandVectorize, CompressVectorize, BlendedLoadVectorize

https://github.com/llvm/llvm-project/pull/199606


More information about the llvm-commits mailing list