[all-commits] [llvm/llvm-project] 4480a2: [LV][EVL] Emit vp.merge intrinsic to enable out-lo...

[Mel Chen via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 4480a22c2b8587c761a44c4290e3fdd9e4be75d3
      https://github.com/llvm/llvm-project/commit/4480a22c2b8587c761a44c4290e3fdd9e4be75d3
  Author: Mel Chen <mel.chen at sifive.com>
  Date:   2024-11-06 (Wed, 06 Nov 2024)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlan.h
    M llvm/lib/Transforms/Vectorize/VPlanPatternMatch.h
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/test/Transforms/LoopVectorize/RISCV/inloop-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-cond-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-inloop-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-intermediate-store.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vplan-vp-intrinsics-reduction.ll

  Log Message:
  -----------
  [LV][EVL] Emit vp.merge intrinsic to enable out-loop reduction in EVL vectorization. (#101641)

Following #90184, this patch emits vp.merge intrinsic, which is used to
set the inactive lanes in a select operation to the RHS instead of
undef. Currently, it is applied to out-loop reduction for EVL
vectorization.

This patch performs transformation to convert 
  select(header_mask, LHS, RHS)
into
  vp.merge(all-true, LHS, RHS, EVL) 
And always use the predicated reduction select to set the incoming value
of the reduction phi to support out-loop reduction when using tail
folding with EVL.

TODO: Postpone the adjustment of the predicated reduction select to
VPlanTransform. The current adjustment might be too early, which could
lead to a situation where the predicated reduction select is adjusted,
but the EVL recipes cannot be successfully generated during
VPlanTransform.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications