[llvm] [VPlan] Add transform to fold early-exit branches into loops (PR #148404)

[Shih-Po Hung via llvm-commits]

================
@@ -7242,6 +7246,10 @@ DenseMap<const SCEV *, Value *> LoopVectorizationPlanner::executePlan(
   // Regions are dissolved after optimizing for VF and UF, which completely
   // removes unneeded loop regions first.
   VPlanTransforms::dissolveLoopRegions(BestVPlan);
+
+  if (FoldEarlyExitBranchIntoLoop)
----------------
arcbbb wrote:

> Do you have any performance data you could share? I think it would be preferable to not require an option for that, we should do it if profitable.

I don't have performance data available at the moment. This approach is the default in downstream, and there isn't a switch for me to test the alternative implementation.

You make a good point about advanced branch predictors. I believe that keeping branches simple and isolated likely prevents interference between unrelated conditions (e.g. predictable condition and unpredictable condition).

The question then becomes whether this should be the canonical form for early exit loop in the vectorizer.
I believe it should be, as it separates distinct conditions (countable and uncountable condition).
And let later transformations with more target-specific information decide whether to merge them based on the microarchitecture.
However, it's fine without this form, since David demonstrated that later transformations are able to handle this pattern.

>  If you look at the actual code generated for AArch64 CPUs when targeting SVE you'll see that after lowering it does end up with this CFG.

I didn't catch that before. Thanks for pointing it out. It's reassuring that SelectionDAG already handles this without this patch.

> I see you're splitting just before the combined condition (the or), and that's normally after all the other recipes, but I think you'd need a check for that before performing that transform.

Thanks for the reminding! I'll move the splitting point to be positioned after the last instruction with side-effect. That should be safe.

https://github.com/llvm/llvm-project/pull/148404