[llvm] [LV] Don't require scalar epilogue for unsupported IAG with tail (PR #96544)
Florian Hahn via llvm-commits
llvm-commits at lists.llvm.org
Sun Jul 14 12:56:43 PDT 2024
================
@@ -1446,24 +1446,42 @@ class LoopVectorizationCostModel {
/// Returns true if we're required to use a scalar epilogue for at least
/// the final iteration of the original loop.
- bool requiresScalarEpilogue(bool IsVectorizing) const {
+ bool requiresScalarEpilogue(ElementCount VF) const {
if (!isScalarEpilogueAllowed()) {
- LLVM_DEBUG(dbgs() << "LV: Loop does not require scalar epilogue\n");
+ LLVM_DEBUG(dbgs() << "LV: Loop with VF = " << VF
+ << " does not require scalar epilogue\n");
return false;
}
// If we might exit from anywhere but the latch, must run the exiting
// iteration in scalar form.
if (TheLoop->getExitingBlock() != TheLoop->getLoopLatch()) {
- LLVM_DEBUG(
- dbgs() << "LV: Loop requires scalar epilogue: multiple exits\n");
+ LLVM_DEBUG(dbgs() << "LV: Loop with VF = " << VF
+ << " requires scalar epilogue: multiple exits\n");
return true;
}
- if (IsVectorizing && InterleaveInfo.requiresScalarEpilogue()) {
- LLVM_DEBUG(dbgs() << "LV: Loop requires scalar epilogue: "
- "interleaved group requires scalar epilogue\n");
- return true;
+ if (VF.isVector()) {
+ if (InterleaveInfo.requiresScalarEpilogue()) {
+ // Make sure interleaved groups that require scalar epilogue will be
+ // widened.
+ if (any_of(InterleaveInfo.getInterleaveGroups(), [&](auto *Group) {
+ if (!Group->requiresScalarEpilogue())
+ return false;
+
+ Instruction *I = Group->getMember(0);
+ InstWidening Decision = getWideningDecision(I, VF);
+ return Decision == CM_Interleave ||
+ (Decision == CM_Unknown &&
----------------
fhahn wrote:
Is it possible that the decision later changes and the result of `requiresScalarEpilogue` changes? as mentioned for https://github.com/llvm/llvm-project/pull/96681 would it be possible to compute whether a scalar epilogue is needed once and set a flag to avoid it potentially changing later?
https://github.com/llvm/llvm-project/pull/96544
More information about the llvm-commits
mailing list