[all-commits] [llvm/llvm-project] e4df6a: [LV] Vectorize (some) early and multiple exit loops

[Philip Reames via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: e4df6a40dad66e989a4333c11d39cf3ed9635135
      https://github.com/llvm/llvm-project/commit/e4df6a40dad66e989a4333c11d39cf3ed9635135
  Author: Philip Reames <listmail at philipreames.com>
  Date:   2020-12-28 (Mon, 28 Dec 2020)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/test/Transforms/LoopVectorize/control-flow.ll
    M llvm/test/Transforms/LoopVectorize/loop-form.ll
    M llvm/test/Transforms/LoopVectorize/loop-legality-checks.ll

  Log Message:
  -----------
  [LV] Vectorize (some) early and multiple exit loops

This patch is a major step towards supporting multiple exit loops in the vectorizer. This patch on it's own extends the loop forms allowed in two ways:

    single exit loops which are not bottom tested
    multiple exit loops w/ a single exit block reached from all exits and no phis in the exit block (because of LCSSA this implies no values defined in the loop used later)

The restrictions on multiple exit loop structures will be removed in follow up patches; disallowing cases for now makes the code changes smaller and more obvious. As before, we can only handle loops with entirely analyzable exits. Removing that restriction is much harder, and is not part of currently planned efforts.

The basic idea here is that we can force the last iteration to run in the scalar epilogue loop (if we have one). From the definition of SCEV's backedge taken count, we know that no earlier iteration can exit the vector body. As such, we can leave the decision on which exit to be taken to the scalar code and generate a bottom tested vector loop which runs all but the last iteration.

The existing code already had the notion of requiring one iteration in the scalar epilogue, this patch is mainly about generalizing that support slightly, making sure we don't try to use this mechanism when tail folding, and updating the code to reflect the difference between a single exit block and a unique exit block (very mechanical).

Differential Revision: https://reviews.llvm.org/D93317