[all-commits] [llvm/llvm-project] 413a66: [LV, VP]VP intrinsics support for the Loop Vectori...

[Alexey Bataev via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 413a66f33984a4d484ac1ff0ba7c0ac39ffa3095
      https://github.com/llvm/llvm-project/commit/413a66f33984a4d484ac1ff0ba7c0ac39ffa3095
  Author: Alexey Bataev <a.bataev at outlook.com>
  Date:   2024-04-04 (Thu, 04 Apr 2024)

  Changed paths:
    M llvm/include/llvm/Analysis/TargetTransformInfo.h
    M llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
    M llvm/lib/Target/RISCV/RISCVTargetTransformInfo.h
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlan.cpp
    M llvm/lib/Transforms/Vectorize/VPlan.h
    M llvm/lib/Transforms/Vectorize/VPlanAnalysis.cpp
    M llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h
    M llvm/lib/Transforms/Vectorize/VPlanValue.h
    M llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp
    A llvm/test/Transforms/LoopVectorize/PowerPC/vectorize-force-tail-with-evl.ll
    A llvm/test/Transforms/LoopVectorize/PowerPC/vplan-force-tail-with-evl.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/inloop-reduction.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-gather-scatter.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-interleave.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-iv32.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-masked-loadstore.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-no-masking.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-reverse-load-store.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/vectorize-vp-intrinsics.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/vplan-vp-intrinsics.ll
    A llvm/test/Transforms/LoopVectorize/X86/vectorize-force-tail-with-evl.ll
    A llvm/test/Transforms/LoopVectorize/X86/vplan-vp-intrinsics.ll
    A llvm/test/Transforms/LoopVectorize/vectorize-force-tail-with-evl.ll
    A llvm/test/Transforms/LoopVectorize/vplan-force-tail-with-evl.ll

  Log Message:
  -----------
  [LV, VP]VP intrinsics support for the Loop Vectorizer + adding new tail-folding mode using EVL. (#76172)

This patch introduces generating VP intrinsics in the Loop Vectorizer.

Currently the Loop Vectorizer supports vector predication in a very
limited capacity via tail-folding and masked load/store/gather/scatter
intrinsics. However, this does not let architectures with active vector
length predication support take advantage of their capabilities.
Architectures with general masked predication support also can only take
advantage of predication on memory operations. By having a way for the
Loop Vectorizer to generate Vector Predication intrinsics, which (will)
provide a target-independent way to model predicated vector
instructions. These architectures can make better use of their
predication capabilities.

Our first approach (implemented in this patch) builds on top of the
existing tail-folding mechanism in the LV (just adds a new tail-folding
mode using EVL), but instead of generating masked intrinsics for memory
operations it generates VP intrinsics for loads/stores instructions. The
patch adds a new VPlanTransforms to replace the wide header predicate
compare with EVL and updates codegen for load/stores to use VP
store/load with EVL.

Other important part of this approach is how the Explicit Vector Length
is computed. (VP intrinsics define this vector length parameter as
Explicit Vector Length (EVL)). We use an experimental intrinsic
`get_vector_length`, that can be lowered to architecture specific
instruction(s) to compute EVL.

Also, added a new recipe to emit instructions for computing EVL. Using
VPlan in this way will eventually help build and compare VPlans
corresponding to different strategies and alternatives.

Differential Revision: https://reviews.llvm.org/D99750



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications