[all-commits] [llvm/llvm-project] b663e5: [VPlan] Fix header masks in EVL tail folding (#150...
Luke Lau via All-commits
all-commits at lists.llvm.org
Tue Jul 29 20:32:01 PDT 2025
Branch: refs/heads/main
Home: https://github.com/llvm/llvm-project
Commit: b663e563cce2bcf6cf7e15799f0ab1cfc56a8361
https://github.com/llvm/llvm-project/commit/b663e563cce2bcf6cf7e15799f0ab1cfc56a8361
Author: Luke Lau <luke at igalia.com>
Date: 2025-07-30 (Wed, 30 Jul 2025)
Changed paths:
M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
M llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp
M llvm/test/Transforms/LoopVectorize/RISCV/pr88802.ll
M llvm/test/Transforms/LoopVectorize/RISCV/tail-folding-cond-reduction.ll
M llvm/test/Transforms/LoopVectorize/RISCV/uniform-load-store.ll
Log Message:
-----------
[VPlan] Fix header masks in EVL tail folding (#150202)
With EVL tail folding, the EVL may not always be VF on the
second-to-last iteration.
Recipes that have been converted to VP intrinsics via optimizeMaskToEVL
account for this, but recipes that are left behind will still use the
old header mask which may end up having a different vector length.
This is effectively the same as #95368, and fixes this by converting
header masks from icmp ule wide-canonical-iv, backedge-trip-count ->
icmp ult step-vector, evl. Without it, recipes that fall through
optimizeMaskToEVL may use the wrong vector length, e.g. in #150074 and
#149981.
We really need to split off optimizeMaskToEVL into
VPlanTransforms::optimize and move transformRecipestoEVLRecipes into
tryToBuildVPlanWithVPRecipes, so we don't mix up what is needed for
correctness and what is needed to optimize away the mask computations.
We should be able to still generate a correct albeit suboptimal VPlan
without running optimizeMaskToEVL. I've added a TODO for this, which I
think we can do after #148274
Fixes #150197
To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications
More information about the All-commits
mailing list