[PATCH] D142875: [LV] Predicated epilog vectorization

[Dave Green via Phabricator via llvm-commits]

dmgreen created this revision.
dmgreen added reviewers: bmahjour, fhahn, Ayal, SjoerdMeijer, sdesmalen, david-arm.
Herald added subscribers: StephenFan, rogfer01, hiraditya.
Herald added a project: All.
dmgreen requested review of this revision.
Herald added subscribers: pcwang-thead, vkmr.
Herald added a project: LLVM.

This extends the creation of epilogs in loop vectorization to include the ability to generate predicated loops from plans that are FoldTailByMasking. Providing that the main loop is unpredicated and there are FoldTailByMasking plans available, it can be profitable to pick one providing they are quicker than a scalar loop (and has a smaller or equal VF to the main body).

The iter.check min epliog iter check and the vec.epilog.iter.check are both changed to always jump to the eplog loop. It is otherwise fairly straight forward, although some of the details may change as we start to use them.


https://reviews.llvm.org/D142875

Files:
  llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
  llvm/lib/Transforms/Vectorize/VPlan.cpp
  llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect.ll
  llvm/test/Transforms/LoopVectorize/ARM/mve-epilogs.ll
  llvm/test/Transforms/LoopVectorize/ARM/tail-folding-reduces-vf.ll
  llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization-scalable.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D142875.493155.patch
Type: text/x-patch
Size: 37298 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230130/bd1d004c/attachment.bin>