[all-commits] [llvm/llvm-project] 97687b: [VPlan] Add active-lane-mask as VPlan-to-VPlan tra...

[Florian Hahn via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 97687b7aea17cfc6e9c42d89b6f22ac5b9c7ddc1
      https://github.com/llvm/llvm-project/commit/97687b7aea17cfc6e9c42d89b6f22ac5b9c7ddc1
  Author: Florian Hahn <flo at fhahn.com>
  Date:   2023-09-25 (Mon, 25 Sep 2023)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlan.cpp
    M llvm/lib/Transforms/Vectorize/VPlan.h
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h
    M llvm/test/Transforms/LoopVectorize/AArch64/scalable-strict-fadd.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-interleaved-masked-accesses.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-forced.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-overflow-checks.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-reductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-unroll.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/tail-fold-uniform-memops.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/tail-folding-styles.ll

  Log Message:
  -----------
  [VPlan] Add active-lane-mask as VPlan-to-VPlan transformation.

This patch updates the mask creation code to always create compares of
the form (ICMP_ULE, wide canonical IV, backedge-taken-count) up front
when tail folding and introduce active-lane-mask as later
transformation.

This effectively makes (ICMP_ULE, wide canonical IV, backedge-taken-count)
the canonical form for tail-folding early on. Introducing more specific
active-lane-mask recipes is treated as a VPlan-to-VPlan optimization.

This has the advantage of keeping the logic  (and complexity) of
introducing active-lane-mask recipes in a single place, instead of
spreading the logic out across multiple functions. It also simplifies
initial VPlan construction and enables treating introducing EVL as
similar optimization.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D158779