[all-commits] [llvm/llvm-project] 723144: [LV] Unconditionally branch from middle to scalar ...

[Philip Reames via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 723144665b7f7afe1555e69eeb3c705c8044d7d0
      https://github.com/llvm/llvm-project/commit/723144665b7f7afe1555e69eeb3c705c8044d7d0
  Author: Philip Reames <listmail at philipreames.com>
  Date:   2021-07-07 (Wed, 07 Jul 2021)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/test/Transforms/LoopVectorize/ARM/mve-gather-scatter-tailpred.ll
    M llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll
    M llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll
    M llvm/test/Transforms/LoopVectorize/loop-form.ll
    M llvm/test/Transforms/LoopVectorize/unroll_nonlatch.ll

  Log Message:
  -----------
  [LV] Unconditionally branch from middle to scalar preheader if the scalar loop must execute (try 4)

Resubmit after the following changes:

* Fix a latent bug related to unrolling with required epilogue (see e49d65f). I believe this is the cause of the prior PPC buildbot failure.
* Disable non-latch exits for epilogue vectorization to be safe (9ffa90d)
* Split out assert movement (600624a) to reduce churn if this gets reverted again.

Previous commit message (try 3)

Resubmit after fixing test/Transforms/LoopVectorize/ARM/mve-gather-scatter-tailpred.ll

Previous commit message...

This is a resubmit of 3e5ce4 (which was reverted by 7fe41ac).  The original commit caused a PPC build bot failure we never really got to the bottom of.  I can't reproduce the issue, and the bot owner was non-responsive.  In the meantime, we stumbled across an issue which seems possibly related, and worked around a latent bug in 80e8025.  My best guess is that the original patch exposed that latent issue at higher frequency, but it really is just a guess.

Original commit message follows...

If we know that the scalar epilogue is required to run, modify the CFG to end the middle block with an unconditional branch to scalar preheader. This is instead of a conditional branch to either the preheader or the exit block.

The motivation to do this is to support multiple exit blocks. Specifically, the current structure forces us to identify immediate dominators and *which* exit block to branch from in the middle terminator. For the multiple exit case - where we know require scalar will hold - these questions are ill formed.

This is the last change needed to support multiple exit loops, but since the diffs are already large enough, I'm going to land this, and then enable separately. You can think of this as being NFCIish prep work, but the changes are a bit too involved for me to feel comfortable tagging the review that way.

Differential Revision: https://reviews.llvm.org/D94892