[all-commits] [llvm/llvm-project] dfca6c: [VPlan] Remove no-op SCALAR-STEPS after unrolling....

Tue Mar 25 05:57:46 PDT 2025

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: dfca6c0d3bf9d1a0565b3ff46813ddd66bd493e9
      https://github.com/llvm/llvm-project/commit/dfca6c0d3bf9d1a0565b3ff46813ddd66bd493e9
  Author: Florian Hahn <flo at fhahn.com>
  Date:   2025-03-25 (Tue, 25 Mar 2025)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/VPlan.h
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/test/Transforms/LoopLoadElim/versioning-scev-invalidation.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/blend-costs.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/clamped-trip-count.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/conditional-branches-cost.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/divs-with-scalable-vfs.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/drop-poison-generating-flags.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/eliminate-tail-predication.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/epilog-vectorization-factors.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/epilog-vectorization-widen-inductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/extractvalue-no-scalarization-required.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/force-target-instruction-cost.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/gather-do-not-vectorize-addressing.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/induction-costs-sve.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/induction-costs.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/interleaved-store-of-first-order-recurrence.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/interleaving-load-store.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/interleaving-reduction.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/loop-vectorization-factors.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/low_trip_count_predicates.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/optsize_minsize.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/partial-reduce-chained.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/partial-reduce-dot-product-epilogue.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/partial-reduce-dot-product-neon.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/partial-reduce-dot-product.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/partial-reduce-sub.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/reduction-recurrence-costs-sve.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/scalable-avoid-scalarization.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/scalable-fp-ext-trunc-illegal-type.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/scalable-reduction-inloop-cond.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/scalable-strict-fadd.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/simple_early_exit.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/store-costs-sve.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/streaming-compatible-sve-no-maximize-bandwidth.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect-inloop-reductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect-reductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect-strict-reductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-epilog-vect.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-inv-store.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-forced.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-optsize.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-reductions.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-widen-gep.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/synthesize-mask-for-call.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/tail-fold-uniform-memops.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/tail-folding-styles.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/transform-narrow-interleave-to-widen-memory-remove-loop-region.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/transform-narrow-interleave-to-widen-memory-with-wide-ops.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/transform-narrow-interleave-to-widen-memory.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/vplan-printing.ll
    M llvm/test/Transforms/LoopVectorize/ARM/mve-gather-scatter-tailpred.ll
    M llvm/test/Transforms/LoopVectorize/ARM/mve-hoist-runtime-checks.ll
    M llvm/test/Transforms/LoopVectorize/ARM/mve-multiexit.ll
    M llvm/test/Transforms/LoopVectorize/ARM/mve-reduction-types.ll
    M llvm/test/Transforms/LoopVectorize/ARM/optsize_minsize.ll
    M llvm/test/Transforms/LoopVectorize/ARM/prefer-tail-loop-folding.ll
    M llvm/test/Transforms/LoopVectorize/ARM/sphinx.ll
    M llvm/test/Transforms/LoopVectorize/ARM/tail-folding-loop-hint.ll
    M llvm/test/Transforms/LoopVectorize/ARM/tail-folding-not-allowed.ll
    M llvm/test/Transforms/LoopVectorize/ARM/tail-folding-prefer-flag.ll
    M llvm/test/Transforms/LoopVectorize/ARM/tail-folding-scalar-epilogue-fallback.ll
    M llvm/test/Transforms/LoopVectorize/PowerPC/exit-branch-cost.ll
    M llvm/test/Transforms/LoopVectorize/PowerPC/large-loop-rdx.ll
    M llvm/test/Transforms/LoopVectorize/PowerPC/optimal-epilog-vectorization.ll
    M llvm/test/Transforms/LoopVectorize/PowerPC/small-loop-rdx.ll
    M llvm/test/Transforms/LoopVectorize/PowerPC/vectorize-bswap.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/bf16.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/blocks-with-dead-instructions.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/dead-ops-cost.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/defaults.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/divrem.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/f16.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/inloop-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/interleaved-accesses.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/lmul.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/low-trip-count.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/mask-index-type.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/ordered-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/pr87378-vpinstruction-or-drop-poison-generating-flags.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/pr88802.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/riscv-unroll.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/riscv-vector-reverse-output.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/riscv-vector-reverse.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/safe-dep-distance.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/scalable-basics.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/scalable-tailfold.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/select-cmp-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/strided-accesses.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/truncate-to-minimal-bitwidth-cost.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/truncate-to-minimal-bitwidth-evl-crash.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/type-info-cache-evl-crash.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/uniform-load-store.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-bin-unary-ops-args.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-call-intrinsics.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-cast-intrinsics.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-cond-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-div.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-fixed-order-recurrence.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-inloop-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-intermediate-store.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-iv32.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-known-no-overflow.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-masked-loadstore.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-ordered-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-reverse-load-store.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-safe-dep-distance.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-uniform-store.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-vp-intrinsics.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vplan-vp-select-intrinsics.ll
    M llvm/test/Transforms/LoopVectorize/WebAssembly/induction-branch-cost.ll
    M llvm/test/Transforms/LoopVectorize/X86/conversion-cost.ll
    M llvm/test/Transforms/LoopVectorize/X86/divs-with-tail-folding.ll
    M llvm/test/Transforms/LoopVectorize/X86/drop-inbounds-flags-for-reverse-vector-pointer.ll
    M llvm/test/Transforms/LoopVectorize/X86/drop-poison-generating-flags.ll
    M llvm/test/Transforms/LoopVectorize/X86/epilog-vectorization-inductions.ll
    M llvm/test/Transforms/LoopVectorize/X86/gather_scatter.ll
    M llvm/test/Transforms/LoopVectorize/X86/gep-use-outside-loop.ll
    M llvm/test/Transforms/LoopVectorize/X86/illegal-parallel-loop-uniform-write.ll
    M llvm/test/Transforms/LoopVectorize/X86/induction-costs.ll
    M llvm/test/Transforms/LoopVectorize/X86/interleave-cost.ll
    M llvm/test/Transforms/LoopVectorize/X86/interleaved-accesses-hoist-load-across-store.ll
    M llvm/test/Transforms/LoopVectorize/X86/interleaved-accesses-sink-store-across-load.ll
    M llvm/test/Transforms/LoopVectorize/X86/limit-vf-by-tripcount.ll
    M llvm/test/Transforms/LoopVectorize/X86/masked-store-cost.ll
    M llvm/test/Transforms/LoopVectorize/X86/masked_load_store.ll
    M llvm/test/Transforms/LoopVectorize/X86/multi-exit-cost.ll
    M llvm/test/Transforms/LoopVectorize/X86/optsize.ll
    M llvm/test/Transforms/LoopVectorize/X86/pr36524.ll
    M llvm/test/Transforms/LoopVectorize/X86/pr47437.ll
    M llvm/test/Transforms/LoopVectorize/X86/pr56319-vector-exit-cond-optimization-epilogue-vectorization.ll
    M llvm/test/Transforms/LoopVectorize/X86/pr81872.ll
    M llvm/test/Transforms/LoopVectorize/X86/predicate-switch.ll
    M llvm/test/Transforms/LoopVectorize/X86/tail_loop_folding.ll
    M llvm/test/Transforms/LoopVectorize/X86/transform-narrow-interleave-to-widen-memory.ll
    M llvm/test/Transforms/LoopVectorize/X86/vect.omp.force.small-tc.ll
    M llvm/test/Transforms/LoopVectorize/X86/vectorize-force-tail-with-evl.ll
    M llvm/test/Transforms/LoopVectorize/X86/vectorize-interleaved-accesses-gap.ll
    M llvm/test/Transforms/LoopVectorize/X86/x86-predication.ll
    M llvm/test/Transforms/LoopVectorize/create-induction-resume.ll
    M llvm/test/Transforms/LoopVectorize/debugloc.ll
    M llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-constant-size.ll
    M llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-variable-size.ll
    M llvm/test/Transforms/LoopVectorize/dont-fold-tail-for-divisible-TC.ll
    M llvm/test/Transforms/LoopVectorize/epilog-iv-select-cmp.ll
    M llvm/test/Transforms/LoopVectorize/epilog-vectorization-any-of-reductions.ll
    M llvm/test/Transforms/LoopVectorize/epilog-vectorization-reductions.ll
    M llvm/test/Transforms/LoopVectorize/epilog-vectorization-trunc-induction-steps.ll
    M llvm/test/Transforms/LoopVectorize/extract-from-end-vector-constant.ll
    M llvm/test/Transforms/LoopVectorize/first-order-recurrence-chains.ll
    M llvm/test/Transforms/LoopVectorize/first-order-recurrence-complex.ll
    M llvm/test/Transforms/LoopVectorize/first-order-recurrence-multiply-recurrences.ll
    M llvm/test/Transforms/LoopVectorize/first-order-recurrence.ll
    M llvm/test/Transforms/LoopVectorize/float-minmax-instruction-flag.ll
    M llvm/test/Transforms/LoopVectorize/fpsat.ll
    M llvm/test/Transforms/LoopVectorize/if-pred-non-void.ll
    M llvm/test/Transforms/LoopVectorize/if-reduction.ll
    M llvm/test/Transforms/LoopVectorize/induction-step.ll
    M llvm/test/Transforms/LoopVectorize/induction.ll
    M llvm/test/Transforms/LoopVectorize/induction_plus.ll
    M llvm/test/Transforms/LoopVectorize/instruction-only-used-outside-of-loop.ll
    M llvm/test/Transforms/LoopVectorize/interleaved-accesses-different-insert-position.ll
    M llvm/test/Transforms/LoopVectorize/invalidate-scev-at-scope-after-vectorization.ll
    M llvm/test/Transforms/LoopVectorize/is_fpclass.ll
    M llvm/test/Transforms/LoopVectorize/iv-select-cmp-nested-loop.ll
    M llvm/test/Transforms/LoopVectorize/iv-select-cmp-no-wrap.ll
    M llvm/test/Transforms/LoopVectorize/iv-select-cmp-trunc.ll
    M llvm/test/Transforms/LoopVectorize/iv-select-cmp.ll
    M llvm/test/Transforms/LoopVectorize/iv_outside_user.ll
    M llvm/test/Transforms/LoopVectorize/load-deref-pred-align.ll
    M llvm/test/Transforms/LoopVectorize/load-of-struct-deref-pred.ll
    M llvm/test/Transforms/LoopVectorize/loop-form.ll
    M llvm/test/Transforms/LoopVectorize/make-followup-loop-id.ll
    M llvm/test/Transforms/LoopVectorize/multiple-strides-vectorization.ll
    M llvm/test/Transforms/LoopVectorize/no-fold-tail-by-masking-iv-external-uses.ll
    M llvm/test/Transforms/LoopVectorize/no_outside_user.ll
    M llvm/test/Transforms/LoopVectorize/opaque-ptr.ll
    M llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization-liveout.ll
    M llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization.ll
    M llvm/test/Transforms/LoopVectorize/optsize.ll
    M llvm/test/Transforms/LoopVectorize/pointer-induction.ll
    M llvm/test/Transforms/LoopVectorize/pr30654-phiscev-sext-trunc.ll
    M llvm/test/Transforms/LoopVectorize/pr35773.ll
    M llvm/test/Transforms/LoopVectorize/pr37248.ll
    M llvm/test/Transforms/LoopVectorize/pr39417-optsize-scevchecks.ll
    M llvm/test/Transforms/LoopVectorize/pr50686.ll
    M llvm/test/Transforms/LoopVectorize/predicate-switch.ll
    M llvm/test/Transforms/LoopVectorize/preserve-or-disjoint.ll
    M llvm/test/Transforms/LoopVectorize/reduction-with-invariant-store.ll
    M llvm/test/Transforms/LoopVectorize/remarks-reduction-inloop.ll
    M llvm/test/Transforms/LoopVectorize/runtime-check-needed-but-empty.ll
    M llvm/test/Transforms/LoopVectorize/runtime-check-small-clamped-bounds.ll
    M llvm/test/Transforms/LoopVectorize/runtime-checks-difference-simplifications.ll
    M llvm/test/Transforms/LoopVectorize/runtime-checks-hoist.ll
    M llvm/test/Transforms/LoopVectorize/scalable-lifetime.ll
    M llvm/test/Transforms/LoopVectorize/scev-exit-phi-invalidation.ll
    M llvm/test/Transforms/LoopVectorize/scev-predicate-reasoning.ll
    M llvm/test/Transforms/LoopVectorize/select-cmp-multiuse.ll
    M llvm/test/Transforms/LoopVectorize/select-cmp.ll
    M llvm/test/Transforms/LoopVectorize/select-min-index.ll
    M llvm/test/Transforms/LoopVectorize/select-reduction-start-value-may-be-undef-or-poison.ll
    M llvm/test/Transforms/LoopVectorize/select-with-fastflags.ll
    M llvm/test/Transforms/LoopVectorize/single-value-blend-phis.ll
    M llvm/test/Transforms/LoopVectorize/single_early_exit.ll
    M llvm/test/Transforms/LoopVectorize/single_early_exit_live_outs.ll
    M llvm/test/Transforms/LoopVectorize/skeleton-lcssa-crash.ll
    M llvm/test/Transforms/LoopVectorize/trip-count-expansion-may-introduce-ub.ll
    M llvm/test/Transforms/LoopVectorize/trunc-extended-icmps.ll
    M llvm/test/Transforms/LoopVectorize/trunc-loads-p16.ll
    M llvm/test/Transforms/LoopVectorize/trunc-shifts.ll
    M llvm/test/Transforms/LoopVectorize/uitofp-preserve-nneg.ll
    M llvm/test/Transforms/LoopVectorize/uniform-blend.ll
    M llvm/test/Transforms/LoopVectorize/uniform_across_vf_induction1.ll
    M llvm/test/Transforms/LoopVectorize/uniform_across_vf_induction1_and.ll
    M llvm/test/Transforms/LoopVectorize/uniform_across_vf_induction1_div_urem.ll
    M llvm/test/Transforms/LoopVectorize/uniform_across_vf_induction1_lshr.ll
    M llvm/test/Transforms/LoopVectorize/uniform_across_vf_induction2.ll
    M llvm/test/Transforms/LoopVectorize/unused-blend-mask-for-first-operand.ll
    M llvm/test/Transforms/LoopVectorize/use-scalar-epilogue-if-tp-fails.ll
    M llvm/test/Transforms/LoopVectorize/vect-phiscev-sext-trunc.ll
    M llvm/test/Transforms/LoopVectorize/vector-intrinsic-call-cost.ll
    M llvm/test/Transforms/LoopVectorize/vector-loop-backedge-elimination-early-exit.ll
    M llvm/test/Transforms/LoopVectorize/vector-loop-backedge-elimination.ll
    M llvm/test/Transforms/LoopVectorize/vectorize-force-tail-with-evl.ll
    M llvm/test/Transforms/LoopVectorize/version-stride-with-integer-casts.ll
    M llvm/test/Transforms/LoopVectorize/widen-gep-all-indices-invariant.ll
    M llvm/test/Transforms/PhaseOrdering/AArch64/sve-interleave-vectorization.ll

  Log Message:
  -----------
  [VPlan] Remove no-op SCALAR-STEPS after unrolling. (#123655)

After unrolling, there may be additional simplifications that can be
applied. One example is removing SCALAR-STEPS for the first part where
only the first lane is demanded.

This removes redundant adds of 0 from a large number of tests (~200),
many which I am still working on updating.

In preparation for removing redundant WideIV steps added in
https://github.com/llvm/llvm-project/pull/119284.

PR: https://github.com/llvm/llvm-project/pull/123655

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications