[llvm] [Transforms][Utils] Add LoopSplitUtils, a reusable iteration-space loop splitter (PR #209142)
via llvm-commits
llvm-commits at lists.llvm.org
Mon Jul 13 04:28:00 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-llvm-transforms
Author: Ashutosh Nema (nema-ashutosh)
<details>
<summary>Changes</summary>
This draft PR presents a patch stack that lays out the overall plan for a general-purpose loop-splitting utility and its adoption.
Its primary focus is introducing LoopSplitUtils as a reusable loop-splitting utility, and then demonstrating how existing passes such as IRCE and LoopBoundSplit can adopt it to perform their transformations.
The patch stack divided into following:
1. **[Transforms][Utils] Add LoopSplitUtils** - the core utility plus a
`loop-split-test` pass (driven from `opt`) and lit coverage. Initial scope:
unit-step, bottom-tested, single-exit loops with a computable trip count.
2. **Constant-step and top-tested loops** - generalize the induction analysis to
any non-zero constant step in either direction and to a counted exit in the
latch or the header.
3. **Loops with multi exits** - Extends support for multiple exit loops
4. **Uncomputable-trip-count and narrow-latch fallbacks** - two opt-in
relaxations (a symbolic counted bound with no exact trip count; a `trunc(iv)`
exit compare), both off by default.
5. **[IRCE]** - add `-irce-use-loop-split-utils` (hidden, default off) to route
IRCE's pre/main/post restructuring through the utility.
6. **[LoopBoundSplit]** - add `-loop-bound-split-use-loop-split-utils` (hidden,
default off) to route the bound split through the utility.
---
Patch is 122.10 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/209142.diff
24 Files Affected:
- (added) llvm/include/llvm/Transforms/Utils/LoopSplitTestPass.h (+29)
- (added) llvm/include/llvm/Transforms/Utils/LoopSplitUtils.h (+209)
- (modified) llvm/lib/Passes/PassBuilder.cpp (+1)
- (modified) llvm/lib/Passes/PassRegistry.def (+1)
- (modified) llvm/lib/Transforms/Scalar/InductiveRangeCheckElimination.cpp (+221-2)
- (modified) llvm/lib/Transforms/Scalar/LoopBoundSplit.cpp (+95)
- (modified) llvm/lib/Transforms/Utils/CMakeLists.txt (+2)
- (added) llvm/lib/Transforms/Utils/LoopSplitTestPass.cpp (+162)
- (added) llvm/lib/Transforms/Utils/LoopSplitUtils.cpp (+852)
- (added) llvm/test/Transforms/IRCE/loop-split-utils-driver-narrow-latch.ll (+39)
- (added) llvm/test/Transforms/IRCE/loop-split-utils-driver.ll (+112)
- (added) llvm/test/Transforms/LoopBoundSplit/loop-split-utils-driver.ll (+51)
- (added) llvm/test/Transforms/LoopSplit/basic.ll (+63)
- (added) llvm/test/Transforms/LoopSplit/descending.ll (+62)
- (added) llvm/test/Transforms/LoopSplit/empty-leading-partition.ll (+73)
- (added) llvm/test/Transforms/LoopSplit/four-partitions.ll (+92)
- (added) llvm/test/Transforms/LoopSplit/multi-exit-inexact-bound-declines.ll (+37)
- (added) llvm/test/Transforms/LoopSplit/multi-exit-inexact-bound.ll (+81)
- (added) llvm/test/Transforms/LoopSplit/multi-exit-reduction.ll (+87)
- (added) llvm/test/Transforms/LoopSplit/multi-exit.ll (+76)
- (added) llvm/test/Transforms/LoopSplit/multiple-partitions.ll (+78)
- (added) llvm/test/Transforms/LoopSplit/optional-guard.ll (+45)
- (added) llvm/test/Transforms/LoopSplit/partition-value-map.ll (+44)
- (added) llvm/test/Transforms/LoopSplit/reduction.ll (+73)
``````````diff
diff --git a/llvm/include/llvm/Transforms/Utils/LoopSplitTestPass.h b/llvm/include/llvm/Transforms/Utils/LoopSplitTestPass.h
new file mode 100644
index 0000000000000..1e427f02a542f
--- /dev/null
+++ b/llvm/include/llvm/Transforms/Utils/LoopSplitTestPass.h
@@ -0,0 +1,29 @@
+//===- LoopSplitTestPass.h - Test driver for LoopSplitUtils -----*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// A command-line driven pass used to exercise the LoopSplitUtils utility from
+// `opt`. The split points are provided via the -loop-split-points option as
+// iteration offsets relative to the induction start.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TRANSFORMS_UTILS_LOOPSPLITTESTPASS_H
+#define LLVM_TRANSFORMS_UTILS_LOOPSPLITTESTPASS_H
+
+#include "llvm/IR/PassManager.h"
+
+namespace llvm {
+
+class LoopSplitTestPass : public PassInfoMixin<LoopSplitTestPass> {
+public:
+ PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM);
+};
+
+} // namespace llvm
+
+#endif // LLVM_TRANSFORMS_UTILS_LOOPSPLITTESTPASS_H
diff --git a/llvm/include/llvm/Transforms/Utils/LoopSplitUtils.h b/llvm/include/llvm/Transforms/Utils/LoopSplitUtils.h
new file mode 100644
index 0000000000000..fa3bf42aaf13d
--- /dev/null
+++ b/llvm/include/llvm/Transforms/Utils/LoopSplitUtils.h
@@ -0,0 +1,209 @@
+//===- LoopSplitUtils.h - Split a loop's iteration space --------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Splits a counted loop's iteration space into a chain of per-partition
+// sub-loops. See LoopSplitUtils.cpp for the structure produced.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TRANSFORMS_UTILS_LOOPSPLITUTILS_H
+#define LLVM_TRANSFORMS_UTILS_LOOPSPLITUTILS_H
+
+#include "llvm/ADT/APInt.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/Support/Compiler.h"
+#include "llvm/Transforms/Utils/ValueMapper.h"
+#include <memory>
+
+namespace llvm {
+
+class BasicBlock;
+class DominatorTree;
+class ICmpInst;
+class Instruction;
+class Loop;
+class LoopInfo;
+class PHINode;
+class SCEV;
+class SCEVAddRecExpr;
+class ScalarEvolution;
+class Value;
+
+/// Splits a counted loop into a chain of per-partition sub-loops.
+///
+/// Usage:
+/// \code
+/// LoopSplitUtils LSU(L, LI, SE, DT);
+/// if (!LSU.isLegal())
+/// return false;
+/// LSU.addPartition(S0, E0); // one call per partition, in order
+/// LSU.addPartition(S1, E1);
+/// LSU.split();
+/// \endcode
+class LoopSplitUtils {
+public:
+ /// \p AllowUncomputableTripCount: also split a multi-exit loop whose counted
+ /// exit has no computable trip count (final partition keeps the original
+ /// latch). Default off.
+ /// \p AllowTruncatedLatchCompare: also split when the counted exit compares a
+ /// truncation of the wider induction (which still drives partitioning). Off.
+ LoopSplitUtils(Loop *L, LoopInfo *LI, ScalarEvolution *SE, DominatorTree *DT,
+ bool AllowUncomputableTripCount = false,
+ bool AllowTruncatedLatchCompare = false)
+ : L(L), LI(LI), SE(SE), DT(DT),
+ AllowUncomputableTripCount(AllowUncomputableTripCount),
+ AllowTruncatedLatchCompare(AllowTruncatedLatchCompare) {}
+
+ /// Analyze \p L; true if it is a counted loop we can split: LCSSA, a counted
+ /// exit in the latch or header, a constant-step integer induction, and an
+ /// exact count. Extra side exits are allowed. Must succeed before split().
+ LLVM_ABI bool isLegal();
+
+ /// Return the loop's induction variable. Valid only after isLegal() succeeds.
+ PHINode *getInductionVariable() const { return Induction; }
+
+ /// Induction value on the final counted iteration (inclusive) -- the end of
+ /// the space the partitions tile. Valid only after isLegal() succeeds.
+ const SCEV *getInductionEnd() const { return InductionEnd; }
+
+ /// True if isLegal() accepted the loop via the uncomputable-trip-count
+ /// fallback (no exact count; the original latch drives the final partition).
+ bool isUncomputableTripCountMode() const { return UncomputableTripCountMode; }
+
+ /// The counted exit's invariant bound, used as the final partition's end in
+ /// uncomputable-trip-count mode. Null unless isUncomputableTripCountMode().
+ const SCEV *getInductionBound() const { return InductionBound; }
+
+ /// Append an inclusive partition range [Start, End] in iteration order.
+ /// Partitions must tile the whole space: first Start = induction start, each
+ /// later Start = previous End +/- step, last End = induction end (desc: S >= E).
+ ///
+ /// Bounds must be loop-invariant and representable in the induction type
+ /// without wrapping: a Start +/- offset that wraps past TYPE_MAX/MIN/0 looks
+ /// in-range and silently miscompiles. See LoopSplitUtils.cpp for the rationale.
+ ///
+ /// Every partition is guarded by default; use avoidPartitionGuard() to opt out.
+ LLVM_ABI void addPartition(const SCEV *Start, const SCEV *End);
+
+ /// Suppress the entry guard for partition \p PartitionIndex (already added). Use
+ /// only for a partition the caller can prove runs at least once; for a runtime-
+ /// empty partition this is incorrect and yields one spurious iteration.
+ LLVM_ABI void avoidPartitionGuard(unsigned PartitionIndex);
+
+ unsigned getNumPartitions() const { return Partitions.size(); }
+
+ /// The loop driving partition \p PartitionIndex: the original loop for 0, the
+ /// clone for a later partition, or null if not cloned. Valid after split().
+ LLVM_ABI Loop *getPartitionLoop(unsigned PartitionIndex) const;
+
+ /// Perform the split. Requires a successful isLegal() and at least two
+ /// partitions. Returns true if the loop was rewritten.
+ LLVM_ABI bool split();
+
+ /// Return the counterpart of original-loop value \p V in partition
+ /// \p PartitionIndex (0-based). Partition 0 maps values to themselves; a later
+ /// partition returns the clone, or null if not cloned. Valid only after split().
+ LLVM_ABI Value *getPartitionValue(const Value *V,
+ unsigned PartitionIndex) const;
+
+ /// Return the original-to-clone value map for the partition at
+ /// \p PartitionIndex, for callers that want to remap many values. Null for
+ /// partition 0 (identity) and for any partition that was not cloned.
+ LLVM_ABI const ValueToValueMapTy *
+ getPartitionValueMap(unsigned PartitionIndex) const;
+
+private:
+ /// Everything known about one partition: the caller-supplied range plus the
+ /// state split() derives. Indexed by partition number in \c Partitions.
+ struct PartitionInfo {
+ // Set by addPartition() / avoidPartitionGuard() before split():
+ const SCEV *StartExpr = nullptr; // inclusive iteration range [Start, End].
+ const SCEV *EndExpr = nullptr;
+ bool Guarded = true; // emit an entry guard?
+
+ // Filled in by split():
+ std::unique_ptr<ValueToValueMapTy> VMap; // null for partition 0 (identity).
+ Value *StartVal = nullptr; // expanded start.
+ Value *SelEnd = nullptr; // clamped end min(End, indEnd).
+ bool Empty = false; // provably zero-iteration.
+ BasicBlock *GuardBlock = nullptr;
+ BasicBlock *Preheader = nullptr;
+ BasicBlock *Exit = nullptr;
+ BasicBlock *ExitingBlk =
+ nullptr; // block holding this partition's exit test.
+ Loop *SubLoop = nullptr;
+ Value *LatchIndOp = nullptr; // induction operand of the exit-test compare.
+ };
+
+ /// Per-split() scratch threaded through the phase helpers (the escaping
+ /// values, new blocks, etc.). A pure transform internal, so it is defined in
+ /// the implementation file.
+ struct SplitState;
+
+ Loop *L;
+ LoopInfo *LI;
+ ScalarEvolution *SE;
+ DominatorTree *DT;
+ bool AllowUncomputableTripCount = false; // opt-in to the fallback below.
+ bool AllowTruncatedLatchCompare = false; // opt-in to a trunc(iv) exit compare.
+
+ // Induction analysis, populated by isLegal().
+ PHINode *Induction = nullptr;
+ BasicBlock *ExitingBlock = nullptr; // the loop's single exiting block.
+ bool IsTopTested = false; // exit test precedes the body (header).
+ ICmpInst *LatchCmp = nullptr; // the exiting block's exit compare.
+ Value *LatchIndOperand = nullptr; // induction operand of the exit compare.
+ bool LatchUsesInductionPHI =
+ false; // exit test compares the PHI, not the step.
+ bool InductionIsSigned = false; // iteration ordering signedness.
+ bool InductionIsDescending = false; // step is negative (loop counts down).
+ APInt InductionStep; // signed constant step, induction width.
+ const SCEV *InductionEnd = nullptr;
+
+ // Uncomputable-trip-count fallback state, set by isLegal() only when there is
+ // no exact count and AllowUncomputableTripCount is set; inert by default.
+ bool UncomputableTripCountMode = false; // accepted via the fallback path.
+ const SCEV *InductionBound =
+ nullptr; // counted compare's invariant bound SCEV.
+ Value *InductionBoundVal = nullptr; // that bound as an IR value (invariant).
+ unsigned CountedContinuePred =
+ 0; // ICmpInst pred P for "IndOp P Bound" == continue.
+
+ /// One record per partition, in add order.
+ SmallVector<PartitionInfo, 4> Partitions;
+
+ /// Find and validate the induction recurrence; returns its add-recurrence, or
+ /// null if the loop has no suitable induction.
+ const SCEVAddRecExpr *analyzeInduction();
+ /// Determine the signedness of the iteration ordering from the latch compare
+ /// and the recurrence's no-wrap flags; returns false if it cannot be proven.
+ bool computeSignedness(const SCEVAddRecExpr *IndAR);
+
+ // split() phase helpers, run in order; each is documented at its definition.
+ /// Collect loop-carried and live-out values and split off the final exit.
+ void collectEscapingValues(SplitState &S);
+ /// Insert the entry guard ahead of partition 0 and update the dominator tree.
+ void buildEntryGuard(SplitState &S);
+ /// Expand each partition's start and clamped end into the entry guard.
+ void expandPartitionBounds(SplitState &S);
+ /// Pass 1: clone each later partition's sub-loop and create its guard/exit.
+ void clonePartitions(SplitState &S);
+ /// Pass 2: emit each guard, clamp each latch, and chain the partitions.
+ void chainPartitions(SplitState &S);
+ /// Rebuild SSA for every escaping value with a per-value SSAUpdater.
+ void reconstructSSA(SplitState &S);
+ /// Clamp \p PL's exit test (in \p ExitingBlk) so it iterates only within
+ /// [start, \p SelEnd], sending the out-of-range edge to \p Exit.
+ void rewriteLatch(Loop *PL, BasicBlock *ExitingBlk, Value *IndOp,
+ Value *SelEnd, BasicBlock *Exit,
+ bool IsLastPartition = false);
+};
+
+} // namespace llvm
+
+#endif // LLVM_TRANSFORMS_UTILS_LOOPSPLITUTILS_H
diff --git a/llvm/lib/Passes/PassBuilder.cpp b/llvm/lib/Passes/PassBuilder.cpp
index 603d7f2f5dea2..4674e4c3c5bd3 100644
--- a/llvm/lib/Passes/PassBuilder.cpp
+++ b/llvm/lib/Passes/PassBuilder.cpp
@@ -368,6 +368,7 @@
#include "llvm/Transforms/Utils/InstructionNamer.h"
#include "llvm/Transforms/Utils/LibCallsShrinkWrap.h"
#include "llvm/Transforms/Utils/LoopSimplify.h"
+#include "llvm/Transforms/Utils/LoopSplitTestPass.h"
#include "llvm/Transforms/Utils/LoopVersioning.h"
#include "llvm/Transforms/Utils/LowerGlobalDtors.h"
#include "llvm/Transforms/Utils/LowerIFunc.h"
diff --git a/llvm/lib/Passes/PassRegistry.def b/llvm/lib/Passes/PassRegistry.def
index 9edb30fedd867..7970434b91dfe 100644
--- a/llvm/lib/Passes/PassRegistry.def
+++ b/llvm/lib/Passes/PassRegistry.def
@@ -481,6 +481,7 @@ FUNCTION_PASS("loop-fusion", LoopFusePass())
FUNCTION_PASS("loop-load-elim", LoopLoadEliminationPass())
FUNCTION_PASS("loop-simplify", LoopSimplifyPass())
FUNCTION_PASS("loop-sink", LoopSinkPass())
+FUNCTION_PASS("loop-split-test", LoopSplitTestPass())
FUNCTION_PASS("loop-versioning", LoopVersioningPass())
FUNCTION_PASS("lower-atomic", LowerAtomicPass())
FUNCTION_PASS("lower-constant-intrinsics", LowerConstantIntrinsicsPass())
diff --git a/llvm/lib/Transforms/Scalar/InductiveRangeCheckElimination.cpp b/llvm/lib/Transforms/Scalar/InductiveRangeCheckElimination.cpp
index 98da1e9225172..d56cd093d34c1 100644
--- a/llvm/lib/Transforms/Scalar/InductiveRangeCheckElimination.cpp
+++ b/llvm/lib/Transforms/Scalar/InductiveRangeCheckElimination.cpp
@@ -67,6 +67,7 @@
#include "llvm/IR/Instructions.h"
#include "llvm/IR/Metadata.h"
#include "llvm/IR/Module.h"
+#include "llvm/IR/Operator.h"
#include "llvm/IR/PatternMatch.h"
#include "llvm/IR/Type.h"
#include "llvm/IR/Use.h"
@@ -83,6 +84,7 @@
#include "llvm/Transforms/Utils/Cloning.h"
#include "llvm/Transforms/Utils/LoopConstrainer.h"
#include "llvm/Transforms/Utils/LoopSimplify.h"
+#include "llvm/Transforms/Utils/LoopSplitUtils.h"
#include "llvm/Transforms/Utils/LoopUtils.h"
#include "llvm/Transforms/Utils/ValueMapper.h"
#include <algorithm>
@@ -126,6 +128,19 @@ static cl::opt<bool>
PrintScaledBoundaryRangeChecks("irce-print-scaled-boundary-range-checks",
cl::Hidden, cl::init(false));
+// When enabled, IRCE restructures the loop's pre/main/post ranges via the
+// generic LoopSplitUtils instead of the bespoke LoopConstrainer -- behaviorally
+// equivalent but structurally different IR. Default off (unchanged output).
+static cl::opt<bool> UseLoopSplitUtils(
+ "irce-use-loop-split-utils", cl::init(false), cl::Hidden,
+ cl::desc(
+ "Drive IRCE's loop restructuring through LoopSplitUtils instead of "
+ "LoopConstrainer where possible"));
+
+// Metadata tag LoopConstrainer stamps on its clones so IRCE skips reprocessing
+// them; the LoopSplitUtils driver reuses it on the pre/post partitions.
+static const char *LoopConstrainerClonedLoopTag = "loop_constrainer.loop.clone";
+
#define DEBUG_TYPE "irce"
namespace {
@@ -985,6 +1000,195 @@ InductiveRangeCheckElimination::estimatedTripCount(const Loop &L) {
return {ExitProbability.scaleByInverse(1)};
}
+// Mark \p L so no further loop optimization runs on it, mirroring
+// LoopConstrainer's DisableAllLoopOptsOnLoop for the pre/post partitions.
+static void disableAllLoopOptsOnLoop(Loop &L) {
+ LLVMContext &Context = L.getHeader()->getContext();
+ MDNode *Dummy = MDNode::get(Context, {});
+ MDNode *DisableUnroll = MDNode::get(
+ Context, {MDString::get(Context, "llvm.loop.unroll.disable")});
+ Metadata *FalseVal =
+ ConstantAsMetadata::get(ConstantInt::get(Type::getInt1Ty(Context), 0));
+ MDNode *DisableVectorize = MDNode::get(
+ Context,
+ {MDString::get(Context, "llvm.loop.vectorize.enable"), FalseVal});
+ MDNode *DisableLICMVersioning = MDNode::get(
+ Context, {MDString::get(Context, "llvm.loop.licm_versioning.disable")});
+ MDNode *DisableDistribution = MDNode::get(
+ Context,
+ {MDString::get(Context, "llvm.loop.distribute.enable"), FalseVal});
+ MDNode *NewLoopID =
+ MDNode::get(Context, {Dummy, DisableUnroll, DisableVectorize,
+ DisableLICMVersioning, DisableDistribution});
+ NewLoopID->replaceOperandWith(0, NewLoopID);
+ L.setLoopID(NewLoopID);
+}
+
+// Restructure \p L into its IRCE sub-ranges via LoopSplitUtils instead of
+// LoopConstrainer, folding eliminated checks in the main partition. The pre/
+// main/post boundaries mirror LoopConstrainer::run exactly (both directions).
+static bool constrainLoopWithLoopSplitUtils(
+ Loop *L, LoopInfo &LI, ScalarEvolution &SE, DominatorTree &DT,
+ const LoopStructure &LS, const LoopConstrainer::SubRanges &SR,
+ Type *RangeTy, function_ref<void(Loop *, bool)> LPMAddNewLoop,
+ SmallVectorImpl<InductiveRangeCheck> &RangeChecksToEliminate) {
+ // Defer a nested loop with an EH-pad side exit: cloning it per partition
+ // re-enters the enclosing loop through the pad, giving irreducible control
+ // flow. (Top-level EH exits leave to non-loop code and stay reducible.)
+ if (L->getParentLoop()) {
+ SmallVector<BasicBlock *, 4> ExitBlocks;
+ L->getExitBlocks(ExitBlocks);
+ for (BasicBlock *Exit : ExitBlocks)
+ if (Exit->isEHPad())
+ return false;
+ }
+
+ // Allow the uncomputable-trip-count fallback (stride>1 symbolic-bound loops
+ // with no computable trip count) and a truncated latch compare (gated by
+ // -irce-allow-narrow-latch, as in calculateSubRanges) so LoopSplitUtils
+ // covers the shapes IRCE constrains.
+ LoopSplitUtils LSU(L, &LI, &SE, &DT, /*AllowUncomputableTripCount=*/true,
+ /*AllowTruncatedLatchCompare=*/AllowNarrowLatchCondition);
+ if (!LSU.isLegal())
+ return false;
+ PHINode *IndPHI = LSU.getInductionVariable();
+ auto *IndTy = dyn_cast<IntegerType>(IndPHI->getType());
+ // The induction is the wide value; require the IRCE range type to match it
+ // (a range check narrower than the latch is rejected by calculateSubRanges).
+ if (!IndTy || IndTy != RangeTy)
+ return false;
+
+ const SCEV *StartS = SE.getSCEV(LS.IndVarStart);
+ if (StartS->getType() != IndTy) {
+ // Narrow-latch: widen LoopStructure's narrower-typed start to the induction/
+ // range type (mirroring LoopConstrainer's NoopOrExtend).
+ auto *StartTy = dyn_cast<IntegerType>(StartS->getType());
+ if (!StartTy || StartTy->getBitWidth() > IndTy->getBitWidth())
+ return false;
+ StartS = NoopOrExtend(StartS, IndTy, SE, LS.IsSignedPredicate);
+ }
+ if (StartS->getType() != IndTy)
+ return false;
+ // Inclusive end of the tiled iteration space: the last counted value in exact
+ // mode; with no computable trip count the invariant bound stands in (the final
+ // partition keeps the original latch, so its end is only a placeholder).
+ const SCEV *EndIncl = LSU.isUncomputableTripCountMode()
+ ? LSU.getInductionBound()
+ : LSU.getInductionEnd();
+ if (!EndIncl || EndIncl->getType() != IndTy)
+ return false;
+
+ bool HasLow = SR.LowLimit.has_value();
+ bool HasHigh = SR.HighLimit.has_value();
+ if (!HasLow && !HasHigh) {
+ // Safe range covers the whole loop: every check is redundant and there is
+ // no head/tail to peel, so no split is needed -- just fold in place. With
+ // nothing to fold, the loop is unchanged.
+ if (RangeChecksToEliminate.empty())
+ return false;
+ LLVMContext &Context = L->getHeader()->getContext();
+ for (InductiveRangeCheck &IRC : RangeChecksToEliminate) {
+ Use *U = IRC.getCheckUse();
+ Value *Folded = IRC.getPassingDirection() ? ConstantInt::getTrue(Context)
+ : ConstantInt::getFalse(Context);
+ U->set(Folded);
+ }
+ return true;
+ }
+
+ const SCEV *One = SE.getOne(IndTy);
+ const SCEV *Low = HasLow ? *SR.LowLimit : nullptr;
+ const SCEV *High = HasHigh ? *SR.HighLimit : nullptr;
+ if ((Low && Low->getType() != IndTy) || (High && High->getType() != IndTy))
+ return false;
+
+ unsigned MainIdx;
+ if (LS.IndVarIncreasing) {
+ const SCEV *MainStart = HasLow ? Low : StartS;
+ const SCEV *MainEnd = HasHigh ? SE.getMinusSCEV(High, One) : EndIncl;
+ if (HasLow)
+ LSU.addPartition(StartS, SE.getMinusSCEV(Low, One));
+ LSU.addPartition(MainStart, MainEnd);
+ if (HasHigh)
+ LSU.addPartition(High, EndIncl);
+ MainIdx = HasLow ? 1 : 0;
+ } else {
+ // Decreasing: LoopSplitUtils carries the runtime induction value across
+ // partitions, so boundaries need not land on the grid. Require a constant
+ // step so the no-overflow reasoning below and the clamp are well defined.
+ const auto *IndAR = dyn_cast<SCEVAddRecExpr>(SE.getSCEV(IndPHI));
+ if (!IndAR)
+ return false;
+ const auto *StepC = dyn_cast<SCEVConstant>(IndAR->getStepRecurrenc...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/209142
More information about the llvm-commits
mailing list