[llvm] [VPlan] Scalarize to first-lane-only directly on VPlan (PR #184267)

[Luke Lau via llvm-commits]

================
@@ -6541,3 +6541,59 @@ void VPlanTransforms::makeMemOpWideningDecisions(
     ReplaceWith(Recipe);
   }
 }
+
+void VPlanTransforms::makeScalarizationDecisions(VPlan &Plan, VFRange &Range) {
+  if (LoopVectorizationPlanner::getDecisionAndClampRange(
+          [&](ElementCount VF) { return VF.isScalar(); }, Range))
+    return;
+
+  // Extend lifetime per `llvm::PostOrderTraversal` documentation:
+  auto PO =
+      vp_post_order_shallow(Plan.getVectorLoopRegion()->getEntryBasicBlock());
+
+  for (VPBasicBlock *VPBB : VPBlockUtils::blocksOnly<VPBasicBlock>(PO)) {
+    for (VPRecipeBase &R : make_early_inc_range(reverse(*VPBB))) {
+      auto *VPI = dyn_cast<VPInstruction>(&R);
+      if (!VPI)
+        continue;
+
+      // For now we (effectively) only scalarize to first-lane-only address
+      // computation chain for the memory operations such that
+      // MemOp->usesFirstLaneOnly(MemOpAddressOperand).
+
+      auto *I = cast_or_null<Instruction>(VPI->getUnderlyingValue());
+      if (!I)
+        // Wouldn't be able to create a `VPReplicateRecipe` anyway.
+        continue;
+
+      // If "executing" other lanes produces side-effects we can't avoid them.
+      if (VPI->mayHaveSideEffects())
+        continue;
+
+      // We want to drop the mask operand, doing that for integer division
+      // isn't safe if it's predicated.
+      if (is_contained({Instruction::SDiv, Instruction::UDiv, Instruction::SRem,
----------------
lukel97 wrote:

Rather than hardcoding the division opcodes it's likely safer to just check `llvm::isSafeToSpeculativelyExecute(I)`. It will also allow cases where the divisor is a known non-zero constant

https://github.com/llvm/llvm-project/pull/184267