[all-commits] [llvm/llvm-project] 51afb1: [LV] Create block in mask up-front if needed. (#76...

[Florian Hahn via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 51afb101743855e2ae2624ebbe087da77128d92c
      https://github.com/llvm/llvm-project/commit/51afb101743855e2ae2624ebbe087da77128d92c
  Author: Florian Hahn <flo at fhahn.com>
  Date:   2024-01-09 (Tue, 09 Jan 2024)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPRecipeBuilder.h
    M llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
    M llvm/test/Transforms/LoopVectorize/AArch64/masked-call.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/scalable-strict-fadd.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-interleaved-masked-accesses.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-reductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-unroll.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/uniform-load-store.ll
    M llvm/test/Transforms/LoopVectorize/X86/drop-poison-generating-flags.ll
    M llvm/test/Transforms/LoopVectorize/X86/imprecise-through-phis.ll
    M llvm/test/Transforms/LoopVectorize/X86/masked_load_store.ll
    M llvm/test/Transforms/LoopVectorize/X86/x86-interleaved-accesses-masked-group.ll
    M llvm/test/Transforms/LoopVectorize/if-conversion-nest.ll
    M llvm/test/Transforms/LoopVectorize/if-pred-non-void.ll
    M llvm/test/Transforms/LoopVectorize/if-reduction.ll
    M llvm/test/Transforms/LoopVectorize/load-of-struct-deref-pred.ll
    M llvm/test/Transforms/LoopVectorize/pr55167-fold-tail-live-out.ll
    M llvm/test/Transforms/LoopVectorize/reduction-inloop-pred.ll
    M llvm/test/Transforms/LoopVectorize/reduction-inloop.ll
    M llvm/test/Transforms/LoopVectorize/reduction.ll
    M llvm/test/Transforms/LoopVectorize/single-value-blend-phis.ll
    M llvm/test/Transforms/LoopVectorize/uniform-blend.ll
    M llvm/test/Transforms/LoopVectorize/vplan-printing.ll
    M llvm/test/Transforms/LoopVectorize/vplan-sink-scalars-and-merge.ll

  Log Message:
  -----------
  [LV] Create block in mask up-front if needed. (#76635)

At the moment, block and edge masks are created on demand, which means
that they are inserted at the point where they are demanded and then
cached. It is possible that the mask for a block is looked up later at a
point that's not dominated by the point where the mask has been
inserted.

To avoid this, create masks up front on entry to the corresponding basic
block and leave it to VPlan simplification to remove unneeded masks.

Note that we need to create masks for all blocks, if any of the blocks
in the loop needs predication, as computing the mask of a block depends
on the masks of its predecessor.

Needed for #76090.

https://github.com/llvm/llvm-project/pull/76635