[all-commits] [llvm/llvm-project] 816097: [LV] Allow tail folded reduction selects to remain...

[David Green via All-commits]

  Branch: refs/heads/master
  Home:   https://github.com/llvm/llvm-project
  Commit: 816097e4e5f32b59c2b6099f74356f1dbe55aabb
      https://github.com/llvm/llvm-project/commit/816097e4e5f32b59c2b6099f74356f1dbe55aabb
  Author: David Green <david.green at arm.com>
  Date:   2020-08-20 (Thu, 20 Aug 2020)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/test/Transforms/LoopVectorize/reduction-predselect.ll

  Log Message:
  -----------
  [LV] Allow tail folded reduction selects to remain in the loop

The normal scheme for tail folding reductions is to use:

loop:
  p = phi(0, a)
  mask = ...
  x = masked_load(..., mask)
  a = add(x, p)
s = select(mask, a, p)

This means we need to keep the register p and a alive out of the loop, plus
the mask. On a target with predicated operations we can instead generate
the phi as p = phi(0, s). This ensures the select in the loop and we can
fold select(m, add(a, b), c) to something like a vaddt c, a, b using the
m predicate. This in turn allows us to tail predicate the entire loop.

Differential Revision: https://reviews.llvm.org/D84741