[llvm] [LoopPeel] Support min/max intrinsics in loop peeling (PR #93162)
Nikita Popov via llvm-commits
llvm-commits at lists.llvm.org
Tue May 28 08:53:52 PDT 2024
================
@@ -436,23 +432,56 @@ static unsigned countToEliminateCompares(Loop &L, unsigned MaxPeelCount,
// However, for equality comparisons, that isn't always sufficient to
// eliminate the comparsion in loop body, we may need to peel one more
// iteration. See if that makes !Pred become unknown again.
+ const SCEV *NextIterVal = SE.getAddExpr(IterVal, Step);
if (ICmpInst::isEquality(Pred) &&
!SE.isKnownPredicate(ICmpInst::getInversePredicate(Pred), NextIterVal,
RightSCEV) &&
!SE.isKnownPredicate(Pred, IterVal, RightSCEV) &&
SE.isKnownPredicate(Pred, NextIterVal, RightSCEV)) {
- if (!CanPeelOneMoreIteration())
+ if (NewPeelCount >= MaxPeelCount)
return; // Need to peel one more iteration, but can't. Give up.
- PeelOneMoreIteration(); // Great!
+ ++NewPeelCount; // Great!
}
DesiredPeelCount = std::max(DesiredPeelCount, NewPeelCount);
};
+ auto ComputePeelCountMinMax = [&](MinMaxIntrinsic *MinMax) {
+ Value *LHS = MinMax->getLHS(), *RHS = MinMax->getRHS();
+ const SCEV *BoundSCEV, *IterSCEV;
+ if (L.isLoopInvariant(LHS)) {
+ BoundSCEV = SE.getSCEV(LHS);
+ IterSCEV = SE.getSCEV(RHS);
+ } else if (L.isLoopInvariant(RHS)) {
+ BoundSCEV = SE.getSCEV(RHS);
+ IterSCEV = SE.getSCEV(LHS);
+ } else
+ return;
+ const auto *AddRec = dyn_cast<SCEVAddRecExpr>(IterSCEV);
+ // For simplicity, we support only affine recurrences.
+ if (!AddRec || !AddRec->isAffine() || AddRec->getLoop() != &L)
+ return;
+ const SCEV *Step = AddRec->getStepRecurrence(SE);
+ // To minimize number of peeled iterations, we use strict relational
+ // predicates here.
+ ICmpInst::Predicate Pred;
+ if (SE.isKnownPositive(Step))
+ Pred = MinMax->isSigned() ? ICmpInst::ICMP_SLT : ICmpInst::ICMP_ULT;
+ else if (SE.isKnownNegative(Step))
+ Pred = MinMax->isSigned() ? ICmpInst::ICMP_SGT : ICmpInst::ICMP_UGT;
+ else
+ return;
+ const SCEV *IterVal = AddRec->evaluateAtIteration(
+ SE.getConstant(AddRec->getType(), DesiredPeelCount), SE);
+ PeelWhilePredicateIsKnown(DesiredPeelCount, IterVal, BoundSCEV, Step, Pred);
----------------
nikic wrote:
Don't we also have to check for a monotonic predicate, as the icmp handling does? Otherwise we might pointlessly peel a wrapping addrec?
https://github.com/llvm/llvm-project/pull/93162
More information about the llvm-commits
mailing list