[llvm] [LV][EVL] Restore inbounds flag in VPVectorEndPointerRecipe when using EVL tail folding (PR #145306)
Mel Chen via llvm-commits
llvm-commits at lists.llvm.org
Tue Jun 24 01:14:15 PDT 2025
================
@@ -2171,6 +2171,29 @@ static VPRecipeBase *createEVLRecipe(VPValue *HeaderMask,
.Default([&](VPRecipeBase *R) { return nullptr; });
}
+// Replace VF operands with EVL in recipes that use VF.
+static void replaceVFWithEVL(VPValue &VF, VPValue &EVL) {
+ for (VPUser *U : to_vector(VF.users())) {
+ if (auto *VEP = dyn_cast<VPVectorEndPointerRecipe>(U)) {
+ VPValue *Ptr = VEP->getPtr();
+ GEPNoWrapFlags NewFlags = VEP->getGEPNoWrapFlags();
+ // Replacing VF with EVL ensures that VPVectorEndPointer will not compute
+ // out-of-bounds, as long as the original pointer had the inbounds flag.
+ if (!NewFlags.isInBounds()) {
+ if (auto *GEP = dyn_cast<GetElementPtrInst>(
----------------
Mel-Chen wrote:
Makes sense. I think we may have two possible approaches to fix this:
1. We can still restore the inbounds flag in VPlanTransforms::tryAddExplicitVectorLength, but only if we verify that all memory access users of the VPVectorEndPointer have had their masks converted to nullptr. However, this still depends on the underlying GEP — if the original GEP doesn't have the inbounds flag, we can't guarantee that the VPVectorEndPointer is inbounds either.
2. Alternatively, we could move this earlier and set the correct flag during VPRecipeBuilder::tryToWidenMemory when CM.foldTailByEVL() is true. Perhaps LoopVectorizationLegality::blockNeedsPredication can help determine whether a memory access is conditional.
This is a side project, and I'm not sure how critical it is. If the cost of implementing an acceptable fix exceeds the above two options, maybe we can defer it.
https://github.com/llvm/llvm-project/pull/145306
More information about the llvm-commits
mailing list