[llvm] [LoopVectorize] Allow Early-Exit Loop Vectorization with EVL (PR #130918)

[David Sherwood via llvm-commits]

================
@@ -4038,10 +4038,12 @@ LoopVectorizationCostModel::computeMaxVF(ElementCount UserVF, unsigned UserIC) {
   }
 
   // The only loops we can vectorize without a scalar epilogue, are loops with
-  // a bottom-test and a single exiting block. We'd have to handle the fact
-  // that not every instruction executes on the last iteration.  This will
-  // require a lane mask which varies through the vector loop body.  (TODO)
-  if (TheLoop->getExitingBlock() != TheLoop->getLoopLatch()) {
+  // a bottom-test and a single exiting block or those with early exits. We'd
+  // have to handle the fact that not every instruction executes on the last
+  // iteration. This will require a lane mask which varies through the vector
+  // loop body. (TODO)
+  if ((TheLoop->getExitingBlock() != TheLoop->getLoopLatch()) &&
+      !Legal->hasUncountableEarlyExit()) {
----------------
david-arm wrote:

This looks like you are in general enabling tail-folding for early exit loops, which I haven't tested at all for AArch64/SVE targets. It may work, but it would be good to add variants of  test/Transforms/LoopVectorize/AArch64/simple_early_exit.ll with tail-folding enabled so that we can verify the correct behaviour. Also, have you run the LLVM test suite with tail-folding and early-exit vectorisation enabled to verify all the tests build and pass? When developing the early exit work I found it was a great test suite for exposing issues. To get better coverage you can modify this code in LoopVectorizationLegality::isVectorizableEarlyExitLoop:

```
  if (!isDereferenceableReadOnlyLoop(TheLoop, PSE.getSE(), DT, AC,
                                     &Predicates)) {
```

to always vectorise:

```
  if (false && !isDereferenceableReadOnlyLoop(TheLoop, PSE.getSE(), DT, AC,
                                     &Predicates)) {
```

It's technically unsafe, but the LLVM test suite is well behaved and tests shouldn't crash.

https://github.com/llvm/llvm-project/pull/130918