[llvm] r347097 - [SimpleLoopUnswitch] adding cost multiplier to cap exponential unswitch with
Hans Wennborg via llvm-commits
llvm-commits at lists.llvm.org
Fri Nov 30 00:55:48 PST 2018
On Fri, Nov 30, 2018 at 9:49 AM Fedor Sergeev <fedor.sergeev at azul.com> wrote:
>
> On 11/29/18 6:29 PM, Hans Wennborg wrote:
> > I noticed this makes use of "sort" in the lit tests. Does this mean
> > there is something non-deterministic in the output that necessitates
> > the sorting? Or is it just that you want to do the checks in a
> > different order than the output would normally be printed?
> Everything is deterministic here.
> The test here does not care about the order of the loops, it just needs
> to check the total
> count, like "10 outer loops, 10 inner loops".
> Sorting allows to group loops of different depths, and that allows to
> apply a simple sequence
> of CHECK-COUNT directives.
Great! Thanks for clarifying.
Cheers,
Hans
> > On Fri, Nov 16, 2018 at 10:19 PM Fedor Sergeev via llvm-commits
> > <llvm-commits at lists.llvm.org> wrote:
> >> Author: fedor.sergeev
> >> Date: Fri Nov 16 13:16:43 2018
> >> New Revision: 347097
> >>
> >> URL: http://llvm.org/viewvc/llvm-project?rev=347097&view=rev
> >> Log:
> >> [SimpleLoopUnswitch] adding cost multiplier to cap exponential unswitch with
> >>
> >> We need to control exponential behavior of loop-unswitch so we do not get
> >> run-away compilation.
> >>
> >> Suggested solution is to introduce a multiplier for an unswitch cost that
> >> makes cost prohibitive as soon as there are too many candidates and too
> >> many sibling loops (meaning we have already started duplicating loops
> >> by unswitching).
> >>
> >> It does solve the currently known problem with compile-time degradation
> >> (PR 39544).
> >>
> >> Tests are built on top of a recently implemented CHECK-COUNT-<num>
> >> FileCheck directives.
> >>
> >> Reviewed By: chandlerc, mkazantsev
> >> Differential Revision: https://reviews.llvm.org/D54223
> >>
> >> Added:
> >> llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested.ll
> >> llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested2.ll
> >> llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch.ll
> >> llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch2.ll
> >> llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-switch-unswitch.ll
> >> Modified:
> >> llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp
> >>
> >> Modified: llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp
> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp?rev=347097&r1=347096&r2=347097&view=diff
> >> ==============================================================================
> >> --- llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp (original)
> >> +++ llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp Fri Nov 16 13:16:43 2018
> >> @@ -62,6 +62,9 @@ STATISTIC(NumBranches, "Number of branch
> >> STATISTIC(NumSwitches, "Number of switches unswitched");
> >> STATISTIC(NumGuards, "Number of guards turned into branches for unswitching");
> >> STATISTIC(NumTrivial, "Number of unswitches that are trivial");
> >> +STATISTIC(
> >> + NumCostMultiplierSkipped,
> >> + "Number of unswitch candidates that had their cost multiplier skipped");
> >>
> >> static cl::opt<bool> EnableNonTrivialUnswitch(
> >> "enable-nontrivial-unswitch", cl::init(false), cl::Hidden,
> >> @@ -72,6 +75,17 @@ static cl::opt<int>
> >> UnswitchThreshold("unswitch-threshold", cl::init(50), cl::Hidden,
> >> cl::desc("The cost threshold for unswitching a loop."));
> >>
> >> +static cl::opt<bool> EnableUnswitchCostMultiplier(
> >> + "enable-unswitch-cost-multiplier", cl::init(true), cl::Hidden,
> >> + cl::desc("Enable unswitch cost multiplier that prohibits exponential "
> >> + "explosion in nontrivial unswitch."));
> >> +static cl::opt<int> UnswitchSiblingsToplevelDiv(
> >> + "unswitch-siblings-toplevel-div", cl::init(2), cl::Hidden,
> >> + cl::desc("Toplevel siblings divisor for cost multiplier."));
> >> +static cl::opt<int> UnswitchNumInitialUnscaledCandidates(
> >> + "unswitch-num-initial-unscaled-candidates", cl::init(8), cl::Hidden,
> >> + cl::desc("Number of unswitch candidates that are ignored when calculating "
> >> + "cost multiplier."));
> >> static cl::opt<bool> UnswitchGuards(
> >> "simple-loop-unswitch-guards", cl::init(true), cl::Hidden,
> >> cl::desc("If enabled, simple loop unswitching will also consider "
> >> @@ -2260,6 +2274,91 @@ turnGuardIntoBranch(IntrinsicInst *GI, L
> >> return CheckBI;
> >> }
> >>
> >> +/// Cost multiplier is a way to limit potentially exponential behavior
> >> +/// of loop-unswitch. Cost is multipied in proportion of 2^number of unswitch
> >> +/// candidates available. Also accounting for the number of "sibling" loops with
> >> +/// the idea to account for previous unswitches that already happened on this
> >> +/// cluster of loops. There was an attempt to keep this formula simple,
> >> +/// just enough to limit the worst case behavior. Even if it is not that simple
> >> +/// now it is still not an attempt to provide a detailed heuristic size
> >> +/// prediction.
> >> +///
> >> +/// TODO: Make a proper accounting of "explosion" effect for all kinds of
> >> +/// unswitch candidates, making adequate predictions instead of wild guesses.
> >> +/// That requires knowing not just the number of "remaining" candidates but
> >> +/// also costs of unswitching for each of these candidates.
> >> +static int calculateUnswitchCostMultiplier(
> >> + Instruction &TI, Loop &L, LoopInfo &LI, DominatorTree &DT,
> >> + ArrayRef<std::pair<Instruction *, TinyPtrVector<Value *>>>
> >> + UnswitchCandidates) {
> >> +
> >> + // Guards and other exiting conditions do not contribute to exponential
> >> + // explosion as soon as they dominate the latch (otherwise there might be
> >> + // another path to the latch remaining that does not allow to eliminate the
> >> + // loop copy on unswitch).
> >> + BasicBlock *Latch = L.getLoopLatch();
> >> + BasicBlock *CondBlock = TI.getParent();
> >> + if (DT.dominates(CondBlock, Latch) &&
> >> + (isGuard(&TI) ||
> >> + llvm::count_if(successors(&TI), [&L](BasicBlock *SuccBB) {
> >> + return L.contains(SuccBB);
> >> + }) <= 1)) {
> >> + NumCostMultiplierSkipped++;
> >> + return 1;
> >> + }
> >> +
> >> + auto *ParentL = L.getParentLoop();
> >> + int SiblingsCount = (ParentL ? ParentL->getSubLoopsVector().size()
> >> + : std::distance(LI.begin(), LI.end()));
> >> + // Count amount of clones that all the candidates might cause during
> >> + // unswitching. Branch/guard counts as 1, switch counts as log2 of its cases.
> >> + int UnswitchedClones = 0;
> >> + for (auto Candidate : UnswitchCandidates) {
> >> + Instruction *CI = Candidate.first;
> >> + BasicBlock *CondBlock = CI->getParent();
> >> + bool SkipExitingSuccessors = DT.dominates(CondBlock, Latch);
> >> + if (isGuard(CI)) {
> >> + if (!SkipExitingSuccessors)
> >> + UnswitchedClones++;
> >> + continue;
> >> + }
> >> + int NonExitingSuccessors = llvm::count_if(
> >> + successors(CondBlock), [SkipExitingSuccessors, &L](BasicBlock *SuccBB) {
> >> + return !SkipExitingSuccessors || L.contains(SuccBB);
> >> + });
> >> + UnswitchedClones += Log2_32(NonExitingSuccessors);
> >> + }
> >> +
> >> + // Ignore up to the "unscaled candidates" number of unswitch candidates
> >> + // when calculating the power-of-two scaling of the cost. The main idea
> >> + // with this control is to allow a small number of unswitches to happen
> >> + // and rely more on siblings multiplier (see below) when the number
> >> + // of candidates is small.
> >> + unsigned ClonesPower =
> >> + std::max(UnswitchedClones - (int)UnswitchNumInitialUnscaledCandidates, 0);
> >> +
> >> + // Allowing top-level loops to spread a bit more than nested ones.
> >> + int SiblingsMultiplier =
> >> + std::max((ParentL ? SiblingsCount
> >> + : SiblingsCount / (int)UnswitchSiblingsToplevelDiv),
> >> + 1);
> >> + // Compute the cost multiplier in a way that won't overflow by saturating
> >> + // at an upper bound.
> >> + int CostMultiplier;
> >> + if (ClonesPower > Log2_32(UnswitchThreshold) ||
> >> + SiblingsMultiplier > UnswitchThreshold)
> >> + CostMultiplier = UnswitchThreshold;
> >> + else
> >> + CostMultiplier = std::min(SiblingsMultiplier * (1 << ClonesPower),
> >> + (int)UnswitchThreshold);
> >> +
> >> + LLVM_DEBUG(dbgs() << " Computed multiplier " << CostMultiplier
> >> + << " (siblings " << SiblingsMultiplier << " * clones "
> >> + << (1 << ClonesPower) << ")"
> >> + << " for unswitch candidate: " << TI << "\n");
> >> + return CostMultiplier;
> >> +}
> >> +
> >> static bool
> >> unswitchBestCondition(Loop &L, DominatorTree &DT, LoopInfo &LI,
> >> AssumptionCache &AC, TargetTransformInfo &TTI,
> >> @@ -2473,8 +2572,23 @@ unswitchBestCondition(Loop &L, Dominator
> >> int CandidateCost = ComputeUnswitchedCost(
> >> TI, /*FullUnswitch*/ !BI || (Invariants.size() == 1 &&
> >> Invariants[0] == BI->getCondition()));
> >> - LLVM_DEBUG(dbgs() << " Computed cost of " << CandidateCost
> >> - << " for unswitch candidate: " << TI << "\n");
> >> + // Calculate cost multiplier which is a tool to limit potentially
> >> + // exponential behavior of loop-unswitch.
> >> + if (EnableUnswitchCostMultiplier) {
> >> + int CostMultiplier =
> >> + calculateUnswitchCostMultiplier(TI, L, LI, DT, UnswitchCandidates);
> >> + assert(
> >> + (CostMultiplier > 0 && CostMultiplier <= UnswitchThreshold) &&
> >> + "cost multiplier needs to be in the range of 1..UnswitchThreshold");
> >> + CandidateCost *= CostMultiplier;
> >> + LLVM_DEBUG(dbgs() << " Computed cost of " << CandidateCost
> >> + << " (multiplier: " << CostMultiplier << ")"
> >> + << " for unswitch candidate: " << TI << "\n");
> >> + } else {
> >> + LLVM_DEBUG(dbgs() << " Computed cost of " << CandidateCost
> >> + << " for unswitch candidate: " << TI << "\n");
> >> + }
> >> +
> >> if (!BestUnswitchTI || CandidateCost < BestUnswitchCost) {
> >> BestUnswitchTI = &TI;
> >> BestUnswitchCost = CandidateCost;
> >>
> >> Added: llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested.ll
> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested.ll?rev=347097&view=auto
> >> ==============================================================================
> >> --- llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested.ll (added)
> >> +++ llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested.ll Fri Nov 16 13:16:43 2018
> >> @@ -0,0 +1,139 @@
> >> +;
> >> +; There should be just a single copy of each loop when strictest mutiplier
> >> +; candidates formula (unscaled candidates == 0) is enforced:
> >> +
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=16 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +;
> >> +; When we relax the candidates part of a multiplier formula
> >> +; (unscaled candidates == 4) we start getting some unswitches,
> >> +; which leads to siblings multiplier kicking in.
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=4 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | \
> >> +; RUN: sort -b | FileCheck %s --check-prefixes=LOOP-UNSCALE4-DIV1
> >> +;
> >> +; NB: sort -b is essential here and below, otherwise blanks might lead to different
> >> +; order depending on locale.
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=4 -unswitch-siblings-toplevel-div=2 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | \
> >> +; RUN: sort -b | FileCheck %s --check-prefixes=LOOP-UNSCALE4-DIV2
> >> +;
> >> +;
> >> +; Get
> >> +; 2^(num conds) == 2^5 = 32
> >> +; loop nests when cost multiplier is disabled:
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=false \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | \
> >> +; RUN: sort -b | FileCheck %s --check-prefixes=LOOP32
> >> +;
> >> +; Single loop nest, not unswitched
> >> +; LOOP1: Loop at depth 1 containing:
> >> +; LOOP1: Loop at depth 2 containing:
> >> +; LOOP1: Loop at depth 3 containing:
> >> +; LOOP1-NOT: Loop at depth {{[0-9]+}} containing:
> >> +;
> >> +; Half unswitched loop nests, with unscaled4 and div1 it gets less depth1 loops unswitched
> >> +; since they have more cost.
> >> +; LOOP-UNSCALE4-DIV1-COUNT-6: Loop at depth 1 containing:
> >> +; LOOP-UNSCALE4-DIV1-COUNT-19: Loop at depth 2 containing:
> >> +; LOOP-UNSCALE4-DIV1-COUNT-29: Loop at depth 3 containing:
> >> +; LOOP-UNSCALE4-DIV1-NOT: Loop at depth {{[0-9]+}} containing:
> >> +;
> >> +; Half unswitched loop nests, with unscaled4 and div2 it gets more depth1 loops unswitched
> >> +; as div2 kicks in.
> >> +; LOOP-UNSCALE4-DIV2-COUNT-11: Loop at depth 1 containing:
> >> +; LOOP-UNSCALE4-DIV2-COUNT-22: Loop at depth 2 containing:
> >> +; LOOP-UNSCALE4-DIV2-COUNT-29: Loop at depth 3 containing:
> >> +; LOOP-UNSCALE4-DIV2-NOT: Loop at depth {{[0-9]+}} containing:
> >> +;
> >> +; 32 loop nests, fully unswitched
> >> +; LOOP32-COUNT-32: Loop at depth 1 containing:
> >> +; LOOP32-COUNT-32: Loop at depth 2 containing:
> >> +; LOOP32-COUNT-32: Loop at depth 3 containing:
> >> +; LOOP32-NOT: Loop at depth {{[0-9]+}} containing:
> >> +
> >> +declare void @bar()
> >> +
> >> +define void @loop_nested3_conds5(i32* %addr, i1 %c1, i1 %c2, i1 %c3, i1 %c4, i1 %c5) {
> >> +entry:
> >> + %addr1 = getelementptr i32, i32* %addr, i64 0
> >> + %addr2 = getelementptr i32, i32* %addr, i64 1
> >> + %addr3 = getelementptr i32, i32* %addr, i64 2
> >> + br label %outer
> >> +outer:
> >> + %iv1 = phi i32 [0, %entry], [%iv1.next, %outer_latch]
> >> + %iv1.next = add i32 %iv1, 1
> >> + ;; skip nontrivial unswitch
> >> + call void @bar()
> >> + br label %middle
> >> +middle:
> >> + %iv2 = phi i32 [0, %outer], [%iv2.next, %middle_latch]
> >> + %iv2.next = add i32 %iv2, 1
> >> + ;; skip nontrivial unswitch
> >> + call void @bar()
> >> + br label %loop
> >> +loop:
> >> + %iv3 = phi i32 [0, %middle], [%iv3.next, %loop_latch]
> >> + %iv3.next = add i32 %iv3, 1
> >> + ;; skip nontrivial unswitch
> >> + call void @bar()
> >> + br i1 %c1, label %loop_next1_left, label %loop_next1_right
> >> +loop_next1_left:
> >> + br label %loop_next1
> >> +loop_next1_right:
> >> + br label %loop_next1
> >> +
> >> +loop_next1:
> >> + br i1 %c2, label %loop_next2_left, label %loop_next2_right
> >> +loop_next2_left:
> >> + br label %loop_next2
> >> +loop_next2_right:
> >> + br label %loop_next2
> >> +
> >> +loop_next2:
> >> + br i1 %c3, label %loop_next3_left, label %loop_next3_right
> >> +loop_next3_left:
> >> + br label %loop_next3
> >> +loop_next3_right:
> >> + br label %loop_next3
> >> +
> >> +loop_next3:
> >> + br i1 %c4, label %loop_next4_left, label %loop_next4_right
> >> +loop_next4_left:
> >> + br label %loop_next4
> >> +loop_next4_right:
> >> + br label %loop_next4
> >> +
> >> +loop_next4:
> >> + br i1 %c5, label %loop_latch_left, label %loop_latch_right
> >> +loop_latch_left:
> >> + br label %loop_latch
> >> +loop_latch_right:
> >> + br label %loop_latch
> >> +
> >> +loop_latch:
> >> + store volatile i32 0, i32* %addr1
> >> + %test_loop = icmp slt i32 %iv3, 50
> >> + br i1 %test_loop, label %loop, label %middle_latch
> >> +middle_latch:
> >> + store volatile i32 0, i32* %addr2
> >> + %test_middle = icmp slt i32 %iv2, 50
> >> + br i1 %test_middle, label %middle, label %outer_latch
> >> +outer_latch:
> >> + store volatile i32 0, i32* %addr3
> >> + %test_outer = icmp slt i32 %iv1, 50
> >> + br i1 %test_outer, label %outer, label %exit
> >> +exit:
> >> + ret void
> >> +}
> >>
> >> Added: llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested2.ll
> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested2.ll?rev=347097&view=auto
> >> ==============================================================================
> >> --- llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested2.ll (added)
> >> +++ llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch-nested2.ll Fri Nov 16 13:16:43 2018
> >> @@ -0,0 +1,149 @@
> >> +;
> >> +; Here all the branches we unswitch are exiting from the inner loop.
> >> +; That means we should not be getting exponential behavior on inner-loop
> >> +; unswitch. In fact there should be just a single version of inner-loop,
> >> +; with possibly some outer loop copies.
> >> +;
> >> +; There should be just a single copy of each loop when strictest mutiplier
> >> +; candidates formula (unscaled candidates == 0) is enforced:
> >> +
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=16 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +;
> >> +; When we relax the candidates part of a multiplier formula
> >> +; (unscaled candidates == 2) we start getting some unswitches in outer loops,
> >> +; which leads to siblings multiplier kicking in.
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=3 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | \
> >> +; RUN: sort -b | FileCheck %s --check-prefixes=LOOP-UNSCALE3-DIV1
> >> +;
> >> +; NB: sort -b is essential here and below, otherwise blanks might lead to different
> >> +; order depending on locale.
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=3 -unswitch-siblings-toplevel-div=2 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | \
> >> +; RUN: sort -b | FileCheck %s --check-prefixes=LOOP-UNSCALE3-DIV2
> >> +;
> >> +; With disabled cost-multiplier we get maximal possible amount of unswitches.
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=false \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | \
> >> +; RUN: sort -b | FileCheck %s --check-prefixes=LOOP-MAX
> >> +;
> >> +; Single loop nest, not unswitched
> >> +; LOOP1: Loop at depth 1 containing:
> >> +; LOOP1-NOT: Loop at depth 1 containing:
> >> +; LOOP1: Loop at depth 2 containing:
> >> +; LOOP1-NOT: Loop at depth 2 containing:
> >> +; LOOP1: Loop at depth 3 containing:
> >> +; LOOP1-NOT: Loop at depth 3 containing:
> >> +;
> >> +; Half unswitched loop nests, with unscaled3 and div1 it gets less depth1 loops unswitched
> >> +; since they have more cost.
> >> +; LOOP-UNSCALE3-DIV1-COUNT-4: Loop at depth 1 containing:
> >> +; LOOP-UNSCALE3-DIV1-NOT: Loop at depth 1 containing:
> >> +; LOOP-UNSCALE3-DIV1-COUNT-1: Loop at depth 2 containing:
> >> +; LOOP-UNSCALE3-DIV1-NOT: Loop at depth 2 containing:
> >> +; LOOP-UNSCALE3-DIV1-COUNT-1: Loop at depth 3 containing:
> >> +; LOOP-UNSCALE3-DIV1-NOT: Loop at depth 3 containing:
> >> +;
> >> +; Half unswitched loop nests, with unscaled3 and div2 it gets more depth1 loops unswitched
> >> +; as div2 kicks in.
> >> +; LOOP-UNSCALE3-DIV2-COUNT-6: Loop at depth 1 containing:
> >> +; LOOP-UNSCALE3-DIV2-NOT: Loop at depth 1 containing:
> >> +; LOOP-UNSCALE3-DIV2-COUNT-1: Loop at depth 2 containing:
> >> +; LOOP-UNSCALE3-DIV2-NOT: Loop at depth 2 containing:
> >> +; LOOP-UNSCALE3-DIV2-COUNT-1: Loop at depth 3 containing:
> >> +; LOOP-UNSCALE3-DIV2-NOT: Loop at depth 3 containing:
> >> +;
> >> +; Maximally unswitched (copy of the outer loop per each condition)
> >> +; LOOP-MAX-COUNT-6: Loop at depth 1 containing:
> >> +; LOOP-MAX-NOT: Loop at depth 1 containing:
> >> +; LOOP-MAX-COUNT-1: Loop at depth 2 containing:
> >> +; LOOP-MAX-NOT: Loop at depth 2 containing:
> >> +; LOOP-MAX-COUNT-1: Loop at depth 3 containing:
> >> +; LOOP-MAX-NOT: Loop at depth 3 containing:
> >> +
> >> +declare void @bar()
> >> +
> >> +define void @loop_nested3_conds5(i32* %addr, i1 %c1, i1 %c2, i1 %c3, i1 %c4, i1 %c5) {
> >> +entry:
> >> + %addr1 = getelementptr i32, i32* %addr, i64 0
> >> + %addr2 = getelementptr i32, i32* %addr, i64 1
> >> + %addr3 = getelementptr i32, i32* %addr, i64 2
> >> + br label %outer
> >> +outer:
> >> + %iv1 = phi i32 [0, %entry], [%iv1.next, %outer_latch]
> >> + %iv1.next = add i32 %iv1, 1
> >> + ;; skip nontrivial unswitch
> >> + call void @bar()
> >> + br label %middle
> >> +middle:
> >> + %iv2 = phi i32 [0, %outer], [%iv2.next, %middle_latch]
> >> + %iv2.next = add i32 %iv2, 1
> >> + ;; skip nontrivial unswitch
> >> + call void @bar()
> >> + br label %loop
> >> +loop:
> >> + %iv3 = phi i32 [0, %middle], [%iv3.next, %loop_latch]
> >> + %iv3.next = add i32 %iv3, 1
> >> + ;; skip nontrivial unswitch
> >> + call void @bar()
> >> + br i1 %c1, label %loop_next1_left, label %outer_latch
> >> +loop_next1_left:
> >> + br label %loop_next1
> >> +loop_next1_right:
> >> + br label %loop_next1
> >> +
> >> +loop_next1:
> >> + br i1 %c2, label %loop_next2_left, label %outer_latch
> >> +loop_next2_left:
> >> + br label %loop_next2
> >> +loop_next2_right:
> >> + br label %loop_next2
> >> +
> >> +loop_next2:
> >> + br i1 %c3, label %loop_next3_left, label %outer_latch
> >> +loop_next3_left:
> >> + br label %loop_next3
> >> +loop_next3_right:
> >> + br label %loop_next3
> >> +
> >> +loop_next3:
> >> + br i1 %c4, label %loop_next4_left, label %outer_latch
> >> +loop_next4_left:
> >> + br label %loop_next4
> >> +loop_next4_right:
> >> + br label %loop_next4
> >> +
> >> +loop_next4:
> >> + br i1 %c5, label %loop_latch_left, label %outer_latch
> >> +loop_latch_left:
> >> + br label %loop_latch
> >> +loop_latch_right:
> >> + br label %loop_latch
> >> +
> >> +loop_latch:
> >> + store volatile i32 0, i32* %addr1
> >> + %test_loop = icmp slt i32 %iv3, 50
> >> + br i1 %test_loop, label %loop, label %middle_latch
> >> +middle_latch:
> >> + store volatile i32 0, i32* %addr2
> >> + %test_middle = icmp slt i32 %iv2, 50
> >> + br i1 %test_middle, label %middle, label %outer_latch
> >> +outer_latch:
> >> + store volatile i32 0, i32* %addr3
> >> + %test_outer = icmp slt i32 %iv1, 50
> >> + br i1 %test_outer, label %outer, label %exit
> >> +exit:
> >> + ret void
> >> +}
> >>
> >> Added: llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch.ll
> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch.ll?rev=347097&view=auto
> >> ==============================================================================
> >> --- llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch.ll (added)
> >> +++ llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch.ll Fri Nov 16 13:16:43 2018
> >> @@ -0,0 +1,80 @@
> >> +;
> >> +; There should be just a single copy of loop when strictest mutiplier candidates
> >> +; formula (unscaled candidates == 0) is enforced:
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=8 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +; With relaxed candidates multiplier (unscaled candidates == 8) we should allow
> >> +; some unswitches to happen until siblings multiplier starts kicking in:
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=8 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP5
> >> +;
> >> +; With relaxed candidates multiplier (unscaled candidates == 8) and with relaxed
> >> +; siblings multiplier for top-level loops (toplevel-div == 8) we should get
> >> +; 2^(num conds) == 2^5 == 32
> >> +; copies of the loop:
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=8 -unswitch-siblings-toplevel-div=8 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP32
> >> +;
> >> +; Similarly get
> >> +; 2^(num conds) == 2^5 == 32
> >> +; copies of the loop when cost multiplier is disabled:
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=false \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP32
> >> +;
> >> +;
> >> +; Single loop, not unswitched
> >> +; LOOP1: Loop at depth 1 containing:
> >> +; LOOP1-NOT: Loop at depth 1 containing:
> >> +
> >> +; 5 loops, unswitched 4 times
> >> +; LOOP5-COUNT-5: Loop at depth 1 containing:
> >> +; LOOP5-NOT: Loop at depth 1 containing:
> >> +
> >> +; 32 loops, fully unswitched
> >> +; LOOP32-COUNT-32: Loop at depth 1 containing:
> >> +; LOOP32-NOT: Loop at depth 1 containing:
> >> +
> >> +define void @loop_simple5(i32* %addr, i1 %c1, i1 %c2, i1 %c3, i1 %c4, i1 %c5) {
> >> +entry:
> >> + br label %loop
> >> +loop:
> >> + %iv = phi i32 [0, %entry], [%iv.next, %loop_latch]
> >> + %iv.next = add i32 %iv, 1
> >> + br i1 %c1, label %loop_next1, label %loop_next1_right
> >> +loop_next1_right:
> >> + br label %loop_next1
> >> +loop_next1:
> >> + br i1 %c2, label %loop_next2, label %loop_next2_right
> >> +loop_next2_right:
> >> + br label %loop_next2
> >> +loop_next2:
> >> + br i1 %c3, label %loop_next3, label %loop_next3_right
> >> +loop_next3_right:
> >> + br label %loop_next3
> >> +loop_next3:
> >> + br i1 %c4, label %loop_next4, label %loop_next4_right
> >> +loop_next4_right:
> >> + br label %loop_next4
> >> +loop_next4:
> >> + br i1 %c5, label %loop_latch, label %loop_latch_right
> >> +loop_latch_right:
> >> + br label %loop_latch
> >> +loop_latch:
> >> + store volatile i32 0, i32* %addr
> >> + %test_loop = icmp slt i32 %iv, 50
> >> + br i1 %test_loop, label %loop, label %exit
> >> +exit:
> >> + ret void
> >> +}
> >>
> >> Added: llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch2.ll
> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch2.ll?rev=347097&view=auto
> >> ==============================================================================
> >> --- llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch2.ll (added)
> >> +++ llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-nontrivial-unswitch2.ll Fri Nov 16 13:16:43 2018
> >> @@ -0,0 +1,56 @@
> >> +;
> >> +; Here all the branches are exiting ones. Checking that we dont have
> >> +; exponential behavior with any kind of controlling heuristics here.
> >> +;
> >> +; There we should have just a single loop.
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=8 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=8 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=8 -unswitch-siblings-toplevel-div=8 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=false \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +;
> >> +; Single loop, not unswitched
> >> +; LOOP1: Loop at depth 1 containing:
> >> +; LOOP1-NOT: Loop at depth 1 containing:
> >> +
> >> +declare void @bar()
> >> +
> >> +define void @loop_simple5(i32* %addr, i1 %c1, i1 %c2, i1 %c3, i1 %c4, i1 %c5) {
> >> +entry:
> >> + br label %loop
> >> +loop:
> >> + %iv = phi i32 [0, %entry], [%iv.next, %loop_latch]
> >> + %iv.next = add i32 %iv, 1
> >> + ;; disabling trivial unswitch
> >> + call void @bar()
> >> + br i1 %c1, label %loop_next1, label %exit
> >> +loop_next1:
> >> + br i1 %c2, label %loop_next2, label %exit
> >> +loop_next2:
> >> + br i1 %c3, label %loop_next3, label %exit
> >> +loop_next3:
> >> + br i1 %c4, label %loop_next4, label %exit
> >> +loop_next4:
> >> + br i1 %c5, label %loop_latch, label %exit
> >> +loop_latch:
> >> + store volatile i32 0, i32* %addr
> >> + %test_loop = icmp slt i32 %iv, 50
> >> + br i1 %test_loop, label %loop, label %exit
> >> +exit:
> >> + ret void
> >> +}
> >>
> >> Added: llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-switch-unswitch.ll
> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-switch-unswitch.ll?rev=347097&view=auto
> >> ==============================================================================
> >> --- llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-switch-unswitch.ll (added)
> >> +++ llvm/trunk/test/Transforms/SimpleLoopUnswitch/exponential-switch-unswitch.ll Fri Nov 16 13:16:43 2018
> >> @@ -0,0 +1,118 @@
> >> +;
> >> +; Here we have 5-way unswitchable switch with each successor also having an unswitchable
> >> +; exiting branch in it. If we start unswitching those branches we start duplicating the
> >> +; whole switch. This can easily lead to exponential behavior w/o proper control.
> >> +; On a real-life testcase there was 16-way switch and that took forever to compile w/o
> >> +; a cost control.
> >> +;
> >> +;
> >> +; When we use the stricted multiplier candidates formula (unscaled candidates == 0)
> >> +; we should be getting just a single loop.
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=0 -unswitch-siblings-toplevel-div=16 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | FileCheck %s --check-prefixes=LOOP1
> >> +;
> >> +;
> >> +; With relaxed candidates multiplier (unscaled candidates == 8) we should allow
> >> +; some unswitches to happen until siblings multiplier starts kicking in:
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=8 -unswitch-siblings-toplevel-div=1 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | \
> >> +; RUN: sort -b | FileCheck %s --check-prefixes=LOOP-RELAX
> >> +;
> >> +; With relaxed candidates multiplier (unscaled candidates == 8) and with relaxed
> >> +; siblings multiplier for top-level loops (toplevel-div == 8) we should get
> >> +; considerably more copies of the loop (especially top-level ones).
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=true \
> >> +; RUN: -unswitch-num-initial-unscaled-candidates=8 -unswitch-siblings-toplevel-div=8 \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | \
> >> +; RUN: sort -b | FileCheck %s --check-prefixes=LOOP-RELAX2
> >> +;
> >> +; We get hundreds of copies of the loop when cost multiplier is disabled:
> >> +;
> >> +; RUN: opt < %s -enable-nontrivial-unswitch -enable-unswitch-cost-multiplier=false \
> >> +; RUN: -passes='loop(unswitch),print<loops>' -disable-output 2>&1 | \
> >> +; RUN: sort -b | FileCheck %s --check-prefixes=LOOP-MAX
> >> +;
> >> +
> >> +; Single loop nest, not unswitched
> >> +; LOOP1: Loop at depth 1 containing:
> >> +; LOOP1-NOT: Loop at depth 1 containing:
> >> +; LOOP1: Loop at depth 2 containing:
> >> +; LOOP1-NOT: Loop at depth 2 containing:
> >> +;
> >> +; Somewhat relaxed restrictions on candidates:
> >> +; LOOP-RELAX-COUNT-5: Loop at depth 1 containing:
> >> +; LOOP-RELAX-NOT: Loop at depth 1 containing:
> >> +; LOOP-RELAX-COUNT-32: Loop at depth 2 containing:
> >> +; LOOP-RELAX-NOT: Loop at depth 2 containing:
> >> +;
> >> +; Even more relaxed restrictions on candidates and siblings.
> >> +; LOOP-RELAX2-COUNT-11: Loop at depth 1 containing:
> >> +; LOOP-RELAX2-NOT: Loop at depth 1 containing:
> >> +; LOOP-RELAX2-COUNT-40: Loop at depth 2 containing:
> >> +; LOOP-RELAX-NOT: Loop at depth 2 containing:
> >> +;
> >> +; Unswitched as much as it could (with multiplier disabled).
> >> +; LOOP-MAX-COUNT-56: Loop at depth 1 containing:
> >> +; LOOP-MAX-NOT: Loop at depth 1 containing:
> >> +; LOOP-MAX-COUNT-111: Loop at depth 2 containing:
> >> +; LOOP-MAX-NOT: Loop at depth 2 containing:
> >> +
> >> +define i32 @loop_switch(i32* %addr, i32 %c1, i32 %c2) {
> >> +entry:
> >> + %addr1 = getelementptr i32, i32* %addr, i64 0
> >> + %addr2 = getelementptr i32, i32* %addr, i64 1
> >> + %check0 = icmp eq i32 %c2, 0
> >> + %check1 = icmp eq i32 %c2, 31
> >> + %check2 = icmp eq i32 %c2, 32
> >> + %check3 = icmp eq i32 %c2, 33
> >> + %check4 = icmp eq i32 %c2, 34
> >> + br label %outer_loop
> >> +
> >> +outer_loop:
> >> + %iv1 = phi i32 [0, %entry], [%iv1.next, %outer_latch]
> >> + %iv1.next = add i32 %iv1, 1
> >> + br label %inner_loop
> >> +inner_loop:
> >> + %iv2 = phi i32 [0, %outer_loop], [%iv2.next, %inner_latch]
> >> + %iv2.next = add i32 %iv2, 1
> >> + switch i32 %c1, label %inner_latch [
> >> + i32 0, label %case0
> >> + i32 1, label %case1
> >> + i32 2, label %case2
> >> + i32 3, label %case3
> >> + i32 4, label %case4
> >> + ]
> >> +
> >> +case4:
> >> + br i1 %check4, label %exit, label %inner_latch
> >> +case3:
> >> + br i1 %check3, label %exit, label %inner_latch
> >> +case2:
> >> + br i1 %check2, label %exit, label %inner_latch
> >> +case1:
> >> + br i1 %check1, label %exit, label %inner_latch
> >> +case0:
> >> + br i1 %check0, label %exit, label %inner_latch
> >> +
> >> +inner_latch:
> >> + store volatile i32 0, i32* %addr1
> >> + %test_inner = icmp slt i32 %iv2, 50
> >> + br i1 %test_inner, label %inner_loop, label %outer_latch
> >> +
> >> +outer_latch:
> >> + store volatile i32 0, i32* %addr2
> >> + %test_outer = icmp slt i32 %iv1, 50
> >> + br i1 %test_outer, label %outer_loop, label %exit
> >> +
> >> +exit: ; preds = %bci_0
> >> + ret i32 1
> >> +}
> >>
> >>
> >> _______________________________________________
> >> llvm-commits mailing list
> >> llvm-commits at lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
More information about the llvm-commits
mailing list