[llvm] [LV] Convert gather loads with constant stride into strided loads (PR #147297)

Mel Chen via llvm-commits llvm-commits at lists.llvm.org
Tue May 5 07:37:17 PDT 2026


================
@@ -6628,3 +6649,111 @@ void VPlanTransforms::makeMemOpWideningDecisions(
     ReplaceWith(Recipe);
   }
 }
+
+void VPlanTransforms::convertToStridedAccesses(VPlan &Plan,
+                                               PredicatedScalarEvolution &PSE,
+                                               Loop &L, VPCostContext &Ctx,
+                                               VFRange &Range) {
+  if (Plan.hasScalarVFOnly())
+    return;
+
+  VPTypeAnalysis TypeInfo(Plan);
+  VPRegionBlock *VectorLoop = Plan.getVectorLoopRegion();
+  SmallVector<VPWidenMemoryRecipe *> ToErase;
+  VPValue *I32VF = nullptr;
+  for (VPBasicBlock *VPBB : VPBlockUtils::blocksOnly<VPBasicBlock>(
+           vp_depth_first_shallow(VectorLoop->getEntry()))) {
+    for (VPRecipeBase &R : make_early_inc_range(*VPBB)) {
+      auto *LoadR = dyn_cast<VPWidenLoadRecipe>(&R);
+      // TODO: Support strided store.
+      // TODO: Transform reverse access into strided access with -1 stride.
+      // TODO: Transform gather/scatter with uniform address into strided access
+      // with 0 stride.
+      // TODO: Transform interleave access into multiple strided accesses.
+      if (!LoadR || LoadR->isConsecutive())
+        continue;
+
+      auto *Ptr = dyn_cast<VPWidenGEPRecipe>(LoadR->getAddr());
+      if (!Ptr)
+        continue;
+
+      Type *LoadTy = TypeInfo.inferScalarType(LoadR);
+      Align Alignment = LoadR->getAlign();
+      auto IsProfitable = [&](ElementCount VF) {
+        Type *DataTy = toVectorTy(LoadTy, VF);
+        if (!Ctx.TTI.isLegalStridedLoadStore(DataTy, Alignment))
+          return false;
+        const InstructionCost CurrentCost = LoadR->computeCost(VF, Ctx);
+        const InstructionCost StridedLoadStoreCost =
+            Ctx.TTI.getMemIntrinsicInstrCost(
+                MemIntrinsicCostAttributes(
+                    Intrinsic::experimental_vp_strided_load, DataTy,
+                    /*Ptr=*/nullptr, LoadR->isMasked(), Alignment),
+                Ctx.CostKind);
+        return StridedLoadStoreCost < CurrentCost;
----------------
Mel-Chen wrote:

7100e99e9b2c29fefd2adcb4abe4c405eabf46e5
I’m going with @fhahn approach for now, though I think @lukel97 's proposal is also good.
However, if we call `::computeCost` directly within `IsProfitable`, we would need to create the recipe via new first, which in turn requires creating its operands. Since most operands for strided accesses require new recipes, does this imply we have to construct the entire recipe chain to evaluate profitability?

https://github.com/llvm/llvm-project/pull/147297


More information about the llvm-commits mailing list