[llvm-branch-commits] [llvm] [SLP] Price gathered lanes of vectorized entries as a shuffle (PR #240136)

Alexey Bataev via llvm-branch-commits llvm-branch-commits at lists.llvm.org
Sun Nov 8 05:24:53 PST 2026


================
@@ -21932,6 +21932,26 @@ BoUpSLP::isGatherShuffledSingleRegisterEntry(
         getScalarizationOverhead(*TTI, Opts.slp_revec, VL.front()->getType(),
                                  MaskVecTy, DemandedElts, /*Insert=*/true,
                                  /*Extract=*/false, CostKind);
+    // A lane taken from another vectorized entry needs a shuffle, which a zero
+    // insert price misses.
+    APInt VectorizedElts = APInt::getZero(SubMask.size());
+    SmallVector<int> VectorizedMask(SubMask.size(), PoisonMaskElem);
+    for (auto [I, Idx] : enumerate(SubMask)) {
+      if (Idx == PoisonMaskElem || !isVectorized(VL[I]))
+        continue;
+      VectorizedElts.setBit(I);
+      VectorizedMask[I] = Idx;
+    }
+    if (!VectorizedElts.isZero() &&
+        getScalarizationOverhead(*TTI, Opts.slp_revec, VL.front()->getType(),
+                                 MaskVecTy, VectorizedElts, /*Insert=*/true,
+                                 /*Extract=*/false,
+                                 CostKind) == TTI::TCC_Free) {
+      InstructionCost VectorizedShuffleCost =
+          GetShuffleCost(VectorizedMask, Entries, VecTy);
----------------
alexey-bataev wrote:

I don't think it is correct. Vectorized mask should be applied to the actual vectorized entries, not ones selected here. Plus, I think you are double-counting reuses.

https://github.com/llvm/llvm-project/pull/240136


More information about the llvm-branch-commits mailing list