[llvm] [VPlan] Model first memory runtime checks as VPlan recipes. (PR #221483)

[Florian Hahn via llvm-commits]

================
@@ -1825,8 +1815,59 @@ class GeneratedRTChecks {
   }
 
   /// Return true if any runtime checks have been added
-  bool hasChecks() const {
-    return getSCEVChecks().first || getMemRuntimeChecks().first;
+  bool hasChecks() const { return getSCEVChecks().first || HasMemChecks; }
+
+  /// Try to generate the memory runtime checks for \p RtPtrChecking as recipes
+  /// in \p Plan, dropping the pre-built block. Returns false if unsupported.
+  /// TODO: Remove the pre-built block once the checks can be costed in VPlan,
+  /// before VF selection.
+  bool
+  tryToAddMemRuntimeChecksToVPlan(VPlan &Plan,
+                                  const RuntimePointerChecking &RtPtrChecking,
+                                  DebugLoc DL, bool AddBranchWeights) {
+    assert(MemCheckBlock && pred_empty(MemCheckBlock) &&
+           "cannot drop memory checks that are missing or already connected");
+    // Diff checks are not modelled in VPlan yet, and the VPlan expander cannot
+    // hoist bounds out of an enclosing loop.
+    if (RtPtrChecking.getDiffChecks() || OuterLoop)
+      return false;
+
+    // VPSCEVExpander expands AddRecs in the plan's entry, not the check block,
+    // and does not support pointer-typed min/max yet.
+    ScalarEvolution &SE = *PSE.getSE();
+    auto IsPtrMinMax = [](const SCEV *S) {
+      return isa<SCEVMinMaxExpr>(S) && S->getType()->isPointerTy();
+    };
+    for (const RuntimeCheckingPtrGroup &CG : RtPtrChecking.CheckingGroups)
+      if (SE.containsAddRecurrence(CG.Low) ||
+          SE.containsAddRecurrence(CG.High) ||
+          SCEVExprContains(CG.Low, IsPtrMinMax) ||
+          SCEVExprContains(CG.High, IsPtrMinMax))
+        return false;
+
+    eraseMemCheckBlock();
+    RUN_VPLAN_PASS(VPlanTransforms::addMemoryRuntimeChecks, Plan,
+                   RtPtrChecking.getChecks(), SE, DL, AddBranchWeights);
+    return true;
+  }
+
+private:
+  /// Erase the memory check block, its instructions and their SCEV expansions.
+  void eraseMemCheckBlock() {
+    SCEVExpanderCleaner MemCheckCleaner(MemCheckExp);
+    auto &SE = *MemCheckExp.getSE();
+    // Memory runtime check generation creates compares that use expanded
+    // values. Remove them before running the SCEVExpanderCleaner.
+    for (auto &I : make_early_inc_range(reverse(*MemCheckBlock))) {
+      if (MemCheckExp.isInsertedInstruction(&I))
+        continue;
+      SE.forgetValue(&I);
+      I.eraseFromParent();
+    }
+    MemCheckCleaner.cleanup();
+    MemCheckBlock->eraseFromParent();
+    MemCheckBlock = nullptr;
+    MemRuntimeCheckCond = nullptr;
----------------
fhahn wrote:

Currently this is unfortuntantely needed due to how the SCEVExpander cleanup/validation works. first we must remove users of the expansion (instructions not inserted by `MemCheckExp`, like the compares), then `MemCheckCleaner` can remove the instructions it created + assert that no users remain.

The cleaner is to enforce all inserted instructions are removed. The SCEVExpander may also insert instructions outside the memcheck block (e.g. because it decided to hoist the expansion).

https://github.com/llvm/llvm-project/pull/221483