[all-commits] [llvm/llvm-project] 0d748b: [LoopVectorize] Extract the last lane from a unifo...

[kmclaughlin-arm via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 0d748b4d32cbddf58a1ff83f3ff178ec1ad49edc
      https://github.com/llvm/llvm-project/commit/0d748b4d32cbddf58a1ff83f3ff178ec1ad49edc
  Author: Kerry McLaughlin <kerry.mclaughlin at arm.com>
  Date:   2021-11-09 (Tue, 09 Nov 2021)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-inv-store.ll
    M llvm/test/Transforms/LoopVectorize/SystemZ/zero_unroll.ll
    M llvm/test/Transforms/LoopVectorize/X86/cost-model-assert.ll
    M llvm/test/Transforms/LoopVectorize/X86/illegal-parallel-loop-uniform-write.ll
    M llvm/test/Transforms/LoopVectorize/X86/uniform_mem_op.ll
    M llvm/test/Transforms/LoopVectorize/pr44488-predication.ll
    M llvm/test/Transforms/LoopVectorize/pr47343-expander-lcssa-after-cfg-update.ll
    M llvm/test/Transforms/LoopVectorize/skeleton-lcssa-crash.ll

  Log Message:
  -----------
  [LoopVectorize] Extract the last lane from a uniform store

Changes VPReplicateRecipe to extract the last lane from an unconditional,
uniform store instruction. collectLoopUniforms will also add stores to
the list of uniform instructions where Legal->isUniformMemOp is true.

setCostBasedWideningDecision now sets the widening decision for
all uniform memory ops to Scalarize, where previously GatherScatter
may have been chosen for scalable stores.

This fixes an assert ("Cannot yet scalarize uniform stores") in
setCostBasedWideningDecision when we have a loop containing a
uniform i1 store and a scalable VF, which we cannot create a scatter for.

Reviewed By: sdesmalen, david-arm, fhahn

Differential Revision: https://reviews.llvm.org/D112725