[all-commits] [llvm/llvm-project] be6e8e: [LV] Tail folded inloop reductions.

[David Green via All-commits]

  Branch: refs/heads/master
  Home:   https://github.com/llvm/llvm-project
  Commit: be6e8e50f4170303e989daea941e8295d9934c66
      https://github.com/llvm/llvm-project/commit/be6e8e50f4170303e989daea941e8295d9934c66
  Author: David Green <david.green at arm.com>
  Date:   2020-10-11 (Sun, 11 Oct 2020)

  Changed paths:
    M llvm/include/llvm/Analysis/IVDescriptors.h
    M llvm/lib/Analysis/IVDescriptors.cpp
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlan.cpp
    M llvm/lib/Transforms/Vectorize/VPlan.h
    M llvm/test/Transforms/LoopVectorize/ARM/mve-gather-scatter-tailpred.ll
    M llvm/test/Transforms/LoopVectorize/ARM/mve-reduction-predselect.ll
    M llvm/test/Transforms/LoopVectorize/ARM/mve-reduction-types.ll
    M llvm/test/Transforms/LoopVectorize/ARM/mve-reductions.ll
    M llvm/test/Transforms/LoopVectorize/X86/optsize.ll
    M llvm/test/Transforms/LoopVectorize/X86/tail_loop_folding.ll
    M llvm/test/Transforms/LoopVectorize/X86/vect.omp.force.small-tc.ll
    M llvm/test/Transforms/LoopVectorize/X86/x86-interleaved-accesses-masked-group.ll
    M llvm/test/Transforms/LoopVectorize/optsize.ll
    M llvm/test/Transforms/LoopVectorize/reduction-inloop-pred.ll
    M llvm/test/Transforms/LoopVectorize/reduction-inloop-uf4.ll
    M llvm/test/Transforms/LoopVectorize/reduction-order.ll

  Log Message:
  -----------
  [LV] Tail folded inloop reductions.

This expands upon the inloop reductions added in e9761688e41cb9e976,
allowing them to be inserted into tail folded loops. Reductions are
generates with the form:

  x = select(mask, vecop, zero)
  v = vecreduce.add(x)
  c = add chain, v

Where zero here is chosen as the identity value for add reductions. The
backend is then expected to fold the select and the vecreduce into a
single predicated instruction.

Most of the code is fairly straight forward, except for the creation of
blockmasks which need to ensure they are created in dominance order. The
order they are added is altered to be after any phis, keeping the
requirements for the underlying IR.

Differential Revision: https://reviews.llvm.org/D84451