[llvm] [LoopVectorize] Enable vectorisation of early exit loops with live-outs (PR #120567)

[David Sherwood via llvm-commits]

================
@@ -9086,19 +9092,38 @@ addUsersInExitBlocks(VPlan &Plan,
       if (Op->isLiveIn())
         continue;
 
-      // Currently only live-ins can be used by exit values from blocks not
-      // exiting via the vector latch through to the middle block.
-      if (ExitIRI->getParent()->getSinglePredecessor() != MiddleVPBB)
-        return false;
-
       LLVMContext &Ctx = ExitIRI->getInstruction().getContext();
-      VPValue *Ext = B.createNaryOp(VPInstruction::ExtractFromEnd,
-                                    {Op, Plan.getOrAddLiveIn(ConstantInt::get(
-                                             IntegerType::get(Ctx, 32), 1))});
+      VPValue *Ext;
+      VPBasicBlock *PredVPBB =
+          cast<VPBasicBlock>(ExitIRI->getParent()->getPredecessors()[Idx]);
----------------
david-arm wrote:

I have implemented your suggestion for handling the live-outs in handleUncountableEarlyExit. I did this in such a way that the LLVM test suite continues to pass when forcing early exit vectorisation and ignoring potentially faulting loads, and also made sure all the tests in llvm/test/Transforms/LoopVectorize pass. There are still ordering issues that need handling delicately, which is why I had to collect the normal exit values prior to calling handleUncountableEarlyExit, otherwise when verifying the IR we crash due to incoming PHI values being matched up with the wrong incoming block.

https://github.com/llvm/llvm-project/pull/120567