[all-commits] [llvm/llvm-project] 2016af: [VPlan] Create epilogue minimum iteration check in...

[Florian Hahn via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 2016af56525734a3ac80d7256434e49e561befc2
      https://github.com/llvm/llvm-project/commit/2016af56525734a3ac80d7256434e49e561befc2
  Author: Florian Hahn <flo at fhahn.com>
  Date:   2025-09-25 (Thu, 25 Sep 2025)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlanConstruction.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h
    M llvm/test/Transforms/LoopVectorize/AArch64/deterministic-type-shrinkage.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/epilog-iv-select-cmp.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/epilog-vectorization-factors.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/epilog-vectorization-widen-inductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/f128-fmuladd-reduction.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/force-target-instruction-cost.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/induction-costs-sve.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/interleave-with-gaps.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/interleaving-load-store.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/interleaving-reduction.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/intrinsiccost.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/loop-vectorization-factors.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/low_trip_count_predicates.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/partial-reduce-dot-product-epilogue.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/reduction-recurrence-costs-sve.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/store-costs-sve.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect-inloop-reductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect-reductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect-strict-reductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vscale-fixed.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve2-histcnt-epilogue.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/transform-narrow-interleave-to-widen-memory-cost.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/vector-loop-backedge-elimination-epilogue.ll
    M llvm/test/Transforms/LoopVectorize/PowerPC/exit-branch-cost.ll
    M llvm/test/Transforms/LoopVectorize/PowerPC/large-loop-rdx.ll
    M llvm/test/Transforms/LoopVectorize/PowerPC/optimal-epilog-vectorization.ll
    M llvm/test/Transforms/LoopVectorize/PowerPC/small-loop-rdx.ll
    M llvm/test/Transforms/LoopVectorize/X86/conversion-cost.ll
    M llvm/test/Transforms/LoopVectorize/X86/cost-conditional-branches.ll
    M llvm/test/Transforms/LoopVectorize/X86/cost-model.ll
    M llvm/test/Transforms/LoopVectorize/X86/epilog-vectorization-inductions.ll
    M llvm/test/Transforms/LoopVectorize/X86/float-induction-x86.ll
    M llvm/test/Transforms/LoopVectorize/X86/gather_scatter.ll
    M llvm/test/Transforms/LoopVectorize/X86/induction-costs.ll
    M llvm/test/Transforms/LoopVectorize/X86/intrinsiccost.ll
    M llvm/test/Transforms/LoopVectorize/X86/invariant-load-gather.ll
    M llvm/test/Transforms/LoopVectorize/X86/invariant-store-vectorization.ll
    M llvm/test/Transforms/LoopVectorize/X86/masked_load_store.ll
    M llvm/test/Transforms/LoopVectorize/X86/pr23997.ll
    M llvm/test/Transforms/LoopVectorize/X86/pr47437.ll
    M llvm/test/Transforms/LoopVectorize/X86/pr54634.ll
    M llvm/test/Transforms/LoopVectorize/X86/scatter_crash.ll
    M llvm/test/Transforms/LoopVectorize/X86/vectorize-force-tail-with-evl.ll
    M llvm/test/Transforms/LoopVectorize/epilog-iv-select-cmp.ll
    M llvm/test/Transforms/LoopVectorize/epilog-vectorization-any-of-reductions.ll
    M llvm/test/Transforms/LoopVectorize/epilog-vectorization-reductions.ll
    M llvm/test/Transforms/LoopVectorize/epilog-vectorization-trunc-induction-steps.ll
    M llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization-liveout.ll
    M llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization.ll
    M llvm/test/Transforms/PhaseOrdering/X86/pr48844-br-to-switch-vectorization.ll
    M llvm/test/Transforms/PhaseOrdering/X86/vdiv.ll

  Log Message:
  -----------
  [VPlan] Create epilogue minimum iteration check in VPlan. (#157545)

Move creation of the minimum iteration check for the epilogue vector
loop to VPlan. This is a first step towards breaking up and moving
skeleton creation for epilogue vectorization to VPlan.

It moves most logic out of EpilogueVectorizerEpilogueLoop: the minimum
iteration check is created directly in VPlan, connecting the check
blocks from the main vector loop is done as post-processing. Next steps
are to move connecting and updating the branches from the check blocks
to VPlan, as well as updating the incoming values for phis.

Test changes are improvements due to folding of live-ins.

PR: https://github.com/llvm/llvm-project/pull/157545



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications