[llvm] [VPlan] Introduce recipes for VP loads and stores. (PR #87816)

[via llvm-commits]

================
@@ -1348,6 +1337,48 @@ void VPlanTransforms::addExplicitVectorLength(VPlan &Plan) {
   CanonicalIVIncrement->setOperand(0, CanonicalIVPHI);
   // TODO: support unroll factor > 1.
   Plan.setUF(1);
+
+  VPValue *BTC = Plan.getOrCreateBackedgeTakenCount();
+  auto IsHeaderMask = [BTC](VPValue *V) {
+    auto *CompareToReplace = dyn_cast<VPInstruction>(V);
+    return CompareToReplace &&
+           CompareToReplace->getOpcode() == Instruction::ICmp &&
+           CompareToReplace->getPredicate() == CmpInst::ICMP_ULE &&
+           CompareToReplace->getOperand(1) == BTC;
----------------
ayalz wrote:

> ... With the new
approach, it is not sufficient to look at users of the widened canonical
IV to find all uses of the header mask.
>
> In some cases, a widened IV is used instead of separately widening the
canonical IV. To handle those cases, iterate over all recipes in the
vector loop region to make sure all widened memory recipes are
processed.

Must every ICMP_ULE of BTC be the header mask, regardless of its other operand?
Should/can the header mask(s) continue to be identified by looking for compares of VPWidenCanonicalIVRecipe that in turn use the canonical IV, before calling `CanonicalIVPHI->replaceAllUsesWith(EVLPhi)`?

Tail folding (an integral part of initial stripmining step according to roadmap) should arguably introduce a single abstract HeaderMask VPValue, retrievable from the loop region (as with its canonical IV), to be later materialized/legalized/lowered into concrete bump/widening/compare/active-lane-mask/no-mask-with-EVL recipes placed inside blocks?

https://github.com/llvm/llvm-project/pull/87816