[llvm] [VPlan] Add exit phi operands during initial construction (NFC). (PR #136455)
via llvm-commits
llvm-commits at lists.llvm.org
Tue Apr 22 13:32:47 PDT 2025
================
@@ -2501,35 +2501,41 @@ void VPlanTransforms::handleUncountableEarlyExit(
if (!ExitIRI)
break;
- PHINode &ExitPhi = ExitIRI->getIRPhi();
- VPValue *IncomingFromEarlyExit = RecipeBuilder.getVPValueOrAddLiveIn(
- ExitPhi.getIncomingValueForBlock(UncountableExitingBlock));
-
+ unsigned EarlyExitIdx = 0;
if (OrigLoop->getUniqueExitBlock()) {
+ // After the transform, the first incoming value is coming from the
+ // orignial loop latch, while the second operand is from the early exit.
+ // Sawp the phi operands, if the first predecessor in the original IR is
+ // not the loop latch.
+ if (*pred_begin(VPEarlyExitBlock->getIRBasicBlock()) !=
+ OrigLoop->getLoopLatch())
+ ExitIRI->swapOperands();
+
+ EarlyExitIdx = 1;
// If there's a unique exit block, VPEarlyExitBlock has 2 predecessors
// (MiddleVPBB and NewMiddle). Add the incoming value from MiddleVPBB
// which is coming from the original latch.
- VPValue *IncomingFromLatch = RecipeBuilder.getVPValueOrAddLiveIn(
- ExitPhi.getIncomingValueForBlock(OrigLoop->getLoopLatch()));
- ExitIRI->addOperand(IncomingFromLatch);
ExitIRI->extractLastLaneOfOperand(MiddleBuilder);
}
+ VPValue *IncomingFromEarlyExit = ExitIRI->getOperand(EarlyExitIdx);
auto IsVector = [](ElementCount VF) { return VF.isVector(); };
// When the VFs are vectors, need to add `extract` to get the incoming value
// from early exit. When the range contains scalar VF, limit the range to
// scalar VF to prevent mis-compilation for the range containing both scalar
// and vector VFs.
if (!IncomingFromEarlyExit->isLiveIn() &&
LoopVectorizationPlanner::getDecisionAndClampRange(IsVector, Range)) {
----------------
ayalz wrote:
(Independent) Still puzzled about clamping the range at this stage, when all VPlans were already created following range clampings. Rather than say asserting that the range contains either scalar or vector VF's but not both, and introduce extracts if it's the latter. Extracts added above for latch exit need not check vector VF's?
https://github.com/llvm/llvm-project/pull/136455
More information about the llvm-commits
mailing list