[PATCH] D109432: [LoopVectorize] Permit fixed-width epilogue loops for scalable vector bodies

[David Sherwood via Phabricator via llvm-commits]

david-arm created this revision.
david-arm added reviewers: sdesmalen, CarolineConcatto, bmahjour, kmclaughlin, fhahn.
Herald added subscribers: ctetreau, rogfer01, bollu, hiraditya, kristof.beyls.
david-arm requested review of this revision.
Herald added subscribers: llvm-commits, vkmr.
Herald added a project: LLVM.

At the moment in LoopVectorizationCostModel::selectEpilogueVectorizationFactor
we bail out if the main vector loop uses a scalable VF. This patch adds
support for generating epilogue vector loops using a fixed-width VF when the
main vector loop uses a scalable VF.

I've changed LoopVectorizationCostModel::selectEpilogueVectorizationFactor
so that we convert the scalable VF into a fixed-width VF and do profitability
checks on that instead. In addition, since the scalable and fixed-width VFs
live in different VPlans that means I had to change the calls to
LVP.hasPlanWithVFs so that we only pass in the fixed-width VF.

Another issue I hit was in LoopVectorizePass::processLoop because we call
setBestPlan for the main vector body, which destroys the plan that we need
for the epilogue. Therefore, I modified setBestPlan to take an optional
pointer to a vector of VPlans where we can keep a backup of the plans we
intend to remove. That way we can restore them again before creating the
epilogue loop.

New tests added here:

  Transforms/LoopVectorize/AArch64/sve-epilog-vect.ll


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D109432

Files:
  llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
  llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
  llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect.ll
  llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization-scalable.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D109432.371301.patch
Type: text/x-patch
Size: 9832 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210908/fc831d14/attachment.bin>