[PATCH] D130364: [LV] Recognize store of invariant value to invariant address as uniform

[Philip Reames via Phabricator via llvm-commits]

reames created this revision.
reames added reviewers: david-arm, fhahn.
Herald added subscribers: frasercrmck, luismarques, apazos, sameer.abuasal, s.egerton, Jim, jocewei, PkmX, the_o, brucehoult, MartinMosbeck, rogfer01, edward-jones, zzheng, jrtc27, niosHD, sabuasal, bollu, simoncook, johnrusso, rbar, asb, hiraditya, mcrosier.
Herald added a project: All.
reames requested review of this revision.
Herald added subscribers: alextsao1999, pcwang-thead, MaskRay.
Herald added a project: LLVM.

This extends the handling of uniform memory operations to handle the case where a store is storing a loop invariant value.  Unlike the general case of a store to an invariant address where we must use the last active lane, in this case we can use any lane since all lanes must produce the same result.

For context, the basic structure of the existing code and how the change fits in:

- First, we select a widening strategy.  (The result is irrelevant for this patch.)
- Then we determine if a computation is uniform within all lanes of VF.  (Note this is the uniform-per-part definition, not LAI's uniform across all unrolled iterations definition.)
- If it is, we overrule the widening strategy, and unconditionally scalarize.
- VPReplicationRecipe - which is what actually does the scalarization - knows how to handle unform-per-part values including for scalable vectors.  However, we do need to know that the expression is safe to execute without predication - e.g. the uniform mem op was unconditional in the original loop.

An obvious question is why not simply implement the generic case?  The answer is that I'm going to, but doing so without a canonicalization towards uniform causes regressions due to bad interaction with scalarization/uniformity of values feeding the uniform mem-op.  This patch is needed to avoid those regressions.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D130364

Files:
  llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
  llvm/test/Transforms/LoopVectorize/RISCV/scalable-tailfold.ll
  llvm/test/Transforms/LoopVectorize/RISCV/uniform-load-store.ll
  llvm/test/Transforms/LoopVectorize/X86/consecutive-ptr-uniforms.ll
  llvm/test/Transforms/LoopVectorize/X86/uniform_mem_op.ll
  llvm/test/Transforms/LoopVectorize/pr45679-fold-tail-by-masking.ll
  llvm/test/Transforms/LoopVectorize/pr46525-expander-insertpoint.ll
  llvm/test/Transforms/LoopVectorize/pr47343-expander-lcssa-after-cfg-update.ll
  llvm/test/Transforms/LoopVectorize/tail-folding-vectorization-factor-1.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D130364.446832.patch
Type: text/x-patch
Size: 46862 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20220722/f89b1f79/attachment.bin>