[PATCH] D152693: LoopVectorize: introduce RecurKind::Induction(I|F)(Max|Min)

[Ramkumar Ramachandra via Phabricator via llvm-commits]

artagnon created this revision.
artagnon added reviewers: dmgreen, fhahn, reames, david-arm, sdesmalen, kmclaughlin.
Herald added subscribers: shiva0217, StephenFan, arphaman, rogfer01, javed.absar, hiraditya.
Herald added a project: All.
artagnon requested review of this revision.
Herald added subscribers: llvm-commits, pcwang-thead, vkmr.
Herald added a project: LLVM.

LoopVectorize's SelectCmp pattern suffers from the deficiency that it
cannot handle non-invariant statements. In order to support
vectorization of the following example,

  int src[n] = {4, 5, 2};
  int r = 331;
  for (int i = 0; i < n; i++) {
    if (src[i] > 3)
      r = i;
  }
  return r;

introduce RecurKind::InductionIMax, RecurKind::InductionIMin,
RecurKind::InductionFMax, and RecurKind::InductionFMin. We currently
only support assignments to the induction variable; in particular, we do
not support the following case:

  int src[n] = {4, 5, 2};
  int r = 331;
  for (int i = 0; i < n; i++) {
    if (src[i] > 3)
      r = src[i];
  }
  return r;

CodeGen'ing for our original example involves checking the SCEV AddRec
expression (the min/max is inverted if the assignment is r = -i, in
place of r = i). Indeed, once we determine whether it's a min or max
reduction, CodeGen'ing involves the following:

1. Create a Splat with the int_min/int_max values, depending on the RecurKind.
2. The Src is filled with values: {0, 1, 2, 3, 4, ...}.
3. The Right is filled with values: {0, 1, 331, 331, 331, ...}.
4. The CmpVector is filled with values: {1, 1, 0, 0, 0, ...}.
5. Select using this CmpVector between Src and the Splat with int_min/int_max values.
6. Use a max-reduce/min-reduce on the result of the Select.
7. Use the exising Cmp which determines whether or not any assignment took place in the loop to select between the result of the max-reduce/min-reduce, and the initial value (InitVal).

Hence, the original example is vectorized.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D152693

Files:
  llvm/include/llvm/Analysis/IVDescriptors.h
  llvm/include/llvm/Transforms/Utils/LoopUtils.h
  llvm/lib/Analysis/IVDescriptors.cpp
  llvm/lib/Transforms/Utils/LoopUtils.cpp
  llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
  llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
  llvm/test/Transforms/LoopVectorize/if-reduction.ll
  llvm/test/Transforms/LoopVectorize/induction-min-max.ll
  llvm/test/Transforms/LoopVectorize/select-min-index.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D152693.530447.patch
Type: text/x-patch
Size: 45435 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230612/e49ee1b6/attachment.bin>