[all-commits] [llvm/llvm-project] 92d092: [VPlan] Fall back to scalar epilogue if possible w...

[Luke Lau via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 92d09245d61dce80d3e68a27cc34d5fc6f062c93
      https://github.com/llvm/llvm-project/commit/92d09245d61dce80d3e68a27cc34d5fc6f062c93
  Author: Luke Lau <luke at igalia.com>
  Date:   2025-07-28 (Mon, 28 Jul 2025)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/test/Transforms/LoopVectorize/RISCV/tail-folding-inloop-reduction.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/tail-folding-reduction.ll

  Log Message:
  -----------
  [VPlan] Fall back to scalar epilogue if possible when EVL isn't legal (#150908)

When enabling predicated vectorization by default on RISC-V, there's a
bunch of performance regressions on llvm-test-suite's LoopInterleaving
microbenchmarks:
https://lnt.lukelau.me/db_default/v4/nts/788?show_delta=yes&show_previous=yes&show_stddev=yes&show_mad=yes&show_all=yes&show_all_samples=yes&show_sample_counts=yes&show_small_diff=yes&num_comparison_runs=0&test_filter=&test_min_value_filter=&aggregation_fn=min&MW_confidence_lv=0.05&compare_to=791&baseline=730&submit=Update

Most of these regressions stem from the interleave_count pragma, which
causes EVL tail folding interleaving to be unsupported (since we don't
support unrolling with EVL)

Currently if DataWithEVL isn't legal we fall back to DataWithoutLaneMask
as the tail folding style, but this is very slow on RISC-V.

The order of performance roughly is something like:

DataWithEVL > None (scalar-epilogue) > Data[WithoutLaneMask]

So this patch tries to prevent the regressions by falling back to a
scalar epilogue where possible, i.e. the existing vectorization we have
today. Not we may still need to fall back to DataWithoutLaneMask, e.g.
if the trip count is low etc or it's forced by
-prefer-predicate-over-epilogue=predicate-dont-vectorize.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications