[PATCH] D146199: [LoopVectorize] Don't tail-fold for scalable VFs when there is no scalar tail

[David Sherwood via Phabricator via llvm-commits]

david-arm created this revision.
david-arm added reviewers: sdesmalen, reames, dtemirbulatov, dmgreen, MattDevereau.
Herald added subscribers: luke, shiva0217, frasercrmck, luismarques, apazos, sameer.abuasal, s.egerton, Jim, jocewei, PkmX, the_o, brucehoult, MartinMosbeck, rogfer01, edward-jones, zzheng, jrtc27, niosHD, sabuasal, simoncook, johnrusso, rbar, asb, hiraditya, arichardson.
Herald added a project: All.
david-arm requested review of this revision.
Herald added subscribers: llvm-commits, pcwang-thead, alextsao1999, MaskRay.
Herald added a project: LLVM.

Currently in LoopVectorize we avoid tail-folding if we can
prove the trip count is always a multiple of the maximum
fixed-width VF. This works because we know the vectoriser
only ever chooses a VF that is a power of 2. However, if
we are also considering scalable VFs then we conservatively
bail out of the optimisation because we don't know the value
of vscale, which could be an odd or prime number, etc.

This patch tries to enable the same optimisation for scalable
VFs by asking if vscale is known to be a power of 2. If so,
we can then query the maximum value of vscale and use the same
logic as we do for fixed-width VFs. I've also added a new TTI
hook called isVScaleKnownToBeAPowerOfTwo that does the same
thing as the existing TargetLowering hook.

I had to change many tests in this patch because they were using
a trip count of 1024, which now triggers this new optimisation
for scalable VFs. However, as a consequence all of the fixed
length tail-folding tests in

  Transforms/LoopVectorize/RISCV/uniform-load-store.ll

now actually attempt to use tail-folding, whereas previously
the optimisation was triggered and they used unpredicated
loops!


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D146199

Files:
  llvm/include/llvm/Analysis/TargetTransformInfo.h
  llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
  llvm/include/llvm/CodeGen/BasicTTIImpl.h
  llvm/lib/Analysis/TargetTransformInfo.cpp
  llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
  llvm/lib/Target/AArch64/AArch64ISelLowering.h
  llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
  llvm/lib/Target/RISCV/RISCVTargetTransformInfo.h
  llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
  llvm/test/Transforms/LoopVectorize/AArch64/masked-call.ll
  llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-optsize.ll
  llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding.ll
  llvm/test/Transforms/LoopVectorize/RISCV/scalable-tailfold.ll
  llvm/test/Transforms/LoopVectorize/RISCV/uniform-load-store.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D146199.505723.patch
Type: text/x-patch
Size: 123573 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230316/7b5b5469/attachment.bin>