<div dir="ltr">I've pushed r336536 that should fix most if not all of the issues here, and I'll continue to do testing to try and see if there is more laying in wait.</div><br><div class="gmail_quote"><div dir="ltr">On Mon, Jul 9, 2018 at 2:51 AM Chandler Carruth <<a href="mailto:chandlerc@gmail.com">chandlerc@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Also <a class="m_-3915603693614047586GWVZpf m_-3915603693614047586gW" id="m_-3915603693614047586IloFPc-2" href="mailto:asbirlea@google.com" target="_blank">+Alina Sbirlea</a> as she's been hacking on top of this...</div><br><div class="gmail_quote"><div dir="ltr">On Mon, Jul 9, 2018 at 2:51 AM Chandler Carruth <<a href="mailto:chandlerc@google.com" target="_blank">chandlerc@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">FYI for folks testing out this functionality, there are collection of serious bugs in this commit. I've got a fix and just need to add some testing. Will have it landed tomorrow. Just wanted to send a heads up.<div><br></div><div><a class="m_-3915603693614047586m_-178678988292647172GWVZpf m_-3915603693614047586m_-178678988292647172gW" id="m_-3915603693614047586m_-178678988292647172IloFPc-0" href="mailto:mikael.holmen@ericsson.com" target="_blank">+Mikael Holmén</a> <a class="m_-3915603693614047586m_-178678988292647172GWVZpf m_-3915603693614047586m_-178678988292647172gW" id="m_-3915603693614047586m_-178678988292647172IloFPc-1" href="mailto:fedor.sergeev@azul.com" target="_blank">+Fedor Sergeev</a> <br></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Jun 25, 2018 at 4:37 PM Chandler Carruth via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: chandlerc<br>
Date: Mon Jun 25 16:32:54 2018<br>
New Revision: 335553<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=335553&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=335553&view=rev</a><br>
Log:<br>
[PM/LoopUnswitch] Teach the new unswitch to handle nontrivial<br>
unswitching of switches.<br>
<br>
This works much like trivial unswitching of switches in that it reliably<br>
moves the switch out of the loop. Here we potentially clone the entire<br>
loop into each successor of the switch and re-point the cases at these<br>
clones.<br>
<br>
Due to the complexity of actually doing nontrivial unswitching, this<br>
patch doesn't create a dedicated routine for handling switches -- it<br>
would duplicate far too much code. Instead, it generalizes the existing<br>
routine to handle both branches and switches as it largely reduces to<br>
looping in a few places instead of doing something once. This actually<br>
improves the results in some cases with branches due to being much more<br>
careful about how dead regions of code are managed. With branches,<br>
because exactly one clone is created and there are exactly two edges<br>
considered, somewhat sloppy handling of the dead regions of code was<br>
sufficient in most cases. But with switches, there are much more<br>
complicated patterns of dead code and so I've had to move to a more<br>
robust model generally. We still do as much pruning of the dead code<br>
early as possible because that allows us to avoid even cloning the code.<br>
<br>
This also surfaced another problem with nontrivial unswitching before<br>
which is that we weren't as precise in reconstructing loops as we could<br>
have been. This seems to have been mostly harmless, but resulted in<br>
pointless LCSSA PHI nodes and other unnecessary cruft. With switches, we<br>
have to get this *right*, and everything benefits from it.<br>
<br>
While the testing may seem a bit light here because we only have two<br>
real cases with actual switches, they do a surprisingly good job of<br>
exercising numerous edge cases. Also, because we share the logic with<br>
branches, most of the changes in this patch are reasonably well covered<br>
by existing tests.<br>
<br>
The new unswitch now has all of the same fundamental power as the old<br>
one with the exception of the single unsound case of *partial* switch<br>
unswitching -- that really is just loop specialization and not<br>
unswitching at all. It doesn't fit into the canonicalization model in<br>
any way. We can add a loop specialization pass that runs late based on<br>
profile data if important test cases ever come up here.<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D47683" rel="noreferrer" target="_blank">https://reviews.llvm.org/D47683</a><br>
<br>
Modified:<br>
llvm/trunk/include/llvm/Transforms/Scalar/SimpleLoopUnswitch.h<br>
llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp<br>
llvm/trunk/test/Transforms/SimpleLoopUnswitch/nontrivial-unswitch.ll<br>
<br>
Modified: llvm/trunk/include/llvm/Transforms/Scalar/SimpleLoopUnswitch.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Scalar/SimpleLoopUnswitch.h?rev=335553&r1=335552&r2=335553&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Scalar/SimpleLoopUnswitch.h?rev=335553&r1=335552&r2=335553&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/include/llvm/Transforms/Scalar/SimpleLoopUnswitch.h (original)<br>
+++ llvm/trunk/include/llvm/Transforms/Scalar/SimpleLoopUnswitch.h Mon Jun 25 16:32:54 2018<br>
@@ -17,9 +17,9 @@<br>
<br>
namespace llvm {<br>
<br>
-/// This pass transforms loops that contain branches on loop-invariant<br>
-/// conditions to have multiple loops. For example, it turns the left into the<br>
-/// right code:<br>
+/// This pass transforms loops that contain branches or switches on loop-<br>
+/// invariant conditions to have multiple loops. For example, it turns the left<br>
+/// into the right code:<br>
///<br>
/// for (...) if (lic)<br>
/// A for (...)<br>
@@ -35,6 +35,31 @@ namespace llvm {<br>
/// This pass expects LICM to be run before it to hoist invariant conditions out<br>
/// of the loop, to make the unswitching opportunity obvious.<br>
///<br>
+/// There is a taxonomy of unswitching that we use to classify different forms<br>
+/// of this transformaiton:<br>
+///<br>
+/// - Trival unswitching: this is when the condition can be unswitched without<br>
+/// cloning any code from inside the loop. A non-trivial unswitch requires<br>
+/// code duplication.<br>
+///<br>
+/// - Full unswitching: this is when the branch or switch is completely moved<br>
+/// from inside the loop to outside the loop. Partial unswitching removes the<br>
+/// branch from the clone of the loop but must leave a (somewhat simplified)<br>
+/// branch in the original loop. While theoretically partial unswitching can<br>
+/// be done for switches, the requirements are extreme - we need the loop<br>
+/// invariant input to the switch to be sufficient to collapse to a single<br>
+/// successor in each clone.<br>
+///<br>
+/// This pass always does trivial, full unswitching for both branches and<br>
+/// switches. For branches, it also always does trivial, partial unswitching.<br>
+///<br>
+/// If enabled (via the constructor's `NonTrivial` parameter), this pass will<br>
+/// additionally do non-trivial, full unswitching for branches and switches, and<br>
+/// will do non-trivial, partial unswitching for branches.<br>
+///<br>
+/// Because partial unswitching of switches is extremely unlikely to be possible<br>
+/// in practice and significantly complicates the implementation, this pass does<br>
+/// not currently implement that in any mode.<br>
class SimpleLoopUnswitchPass : public PassInfoMixin<SimpleLoopUnswitchPass> {<br>
bool NonTrivial;<br>
<br>
<br>
Modified: llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp?rev=335553&r1=335552&r2=335553&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp?rev=335553&r1=335552&r2=335553&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp (original)<br>
+++ llvm/trunk/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp Mon Jun 25 16:32:54 2018<br>
@@ -715,8 +715,12 @@ static bool unswitchAllTrivialConditions<br>
///<br>
/// This routine handles cloning all of the necessary loop blocks and exit<br>
/// blocks including rewriting their instructions and the relevant PHI nodes.<br>
-/// It skips loop and exit blocks that are not necessary based on the provided<br>
-/// set. It also correctly creates the unconditional branch in the cloned<br>
+/// Any loop blocks or exit blocks which are dominated by a different successor<br>
+/// than the one for this clone of the loop blocks can be trivially skipped. We<br>
+/// use the `DominatingSucc` map to determine whether a block satisfies that<br>
+/// property with a simple map lookup.<br>
+///<br>
+/// It also correctly creates the unconditional branch in the cloned<br>
/// unswitched parent block to only point at the unswitched successor.<br>
///<br>
/// This does not handle most of the necessary updates to `LoopInfo`. Only exit<br>
@@ -730,7 +734,7 @@ static BasicBlock *buildClonedLoopBlocks<br>
Loop &L, BasicBlock *LoopPH, BasicBlock *SplitBB,<br>
ArrayRef<BasicBlock *> ExitBlocks, BasicBlock *ParentBB,<br>
BasicBlock *UnswitchedSuccBB, BasicBlock *ContinueSuccBB,<br>
- const SmallPtrSetImpl<BasicBlock *> &SkippedLoopAndExitBlocks,<br>
+ const SmallDenseMap<BasicBlock *, BasicBlock *, 16> &DominatingSucc,<br>
ValueToValueMapTy &VMap,<br>
SmallVectorImpl<DominatorTree::UpdateType> &DTUpdates, AssumptionCache &AC,<br>
DominatorTree &DT, LoopInfo &LI) {<br>
@@ -751,19 +755,26 @@ static BasicBlock *buildClonedLoopBlocks<br>
return NewBB;<br>
};<br>
<br>
+ // We skip cloning blocks when they have a dominating succ that is not the<br>
+ // succ we are cloning for.<br>
+ auto SkipBlock = [&](BasicBlock *BB) {<br>
+ auto It = DominatingSucc.find(BB);<br>
+ return It != DominatingSucc.end() && It->second != UnswitchedSuccBB;<br>
+ };<br>
+<br>
// First, clone the preheader.<br>
auto *ClonedPH = CloneBlock(LoopPH);<br>
<br>
// Then clone all the loop blocks, skipping the ones that aren't necessary.<br>
for (auto *LoopBB : L.blocks())<br>
- if (!SkippedLoopAndExitBlocks.count(LoopBB))<br>
+ if (!SkipBlock(LoopBB))<br>
CloneBlock(LoopBB);<br>
<br>
// Split all the loop exit edges so that when we clone the exit blocks, if<br>
// any of the exit blocks are *also* a preheader for some other loop, we<br>
// don't create multiple predecessors entering the loop header.<br>
for (auto *ExitBB : ExitBlocks) {<br>
- if (SkippedLoopAndExitBlocks.count(ExitBB))<br>
+ if (SkipBlock(ExitBB))<br>
continue;<br>
<br>
// When we are going to clone an exit, we don't need to clone all the<br>
@@ -841,7 +852,7 @@ static BasicBlock *buildClonedLoopBlocks<br>
// Update any PHI nodes in the cloned successors of the skipped blocks to not<br>
// have spurious incoming values.<br>
for (auto *LoopBB : L.blocks())<br>
- if (SkippedLoopAndExitBlocks.count(LoopBB))<br>
+ if (SkipBlock(LoopBB))<br>
for (auto *SuccBB : successors(LoopBB))<br>
if (auto *ClonedSuccBB = cast_or_null<BasicBlock>(VMap.lookup(SuccBB)))<br>
for (PHINode &PN : ClonedSuccBB->phis())<br>
@@ -1175,10 +1186,41 @@ static void buildClonedLoops(Loop &OrigL<br>
}<br>
<br>
static void<br>
+deleteDeadClonedBlocks(Loop &L, ArrayRef<BasicBlock *> ExitBlocks,<br>
+ ArrayRef<std::unique_ptr<ValueToValueMapTy>> VMaps,<br>
+ DominatorTree &DT) {<br>
+ // Find all the dead clones, and remove them from their successors.<br>
+ SmallVector<BasicBlock *, 16> DeadBlocks;<br>
+ for (BasicBlock *BB : llvm::concat<BasicBlock *const>(L.blocks(), ExitBlocks))<br>
+ for (auto &VMap : VMaps)<br>
+ if (BasicBlock *ClonedBB = cast_or_null<BasicBlock>(VMap->lookup(BB)))<br>
+ if (!DT.isReachableFromEntry(ClonedBB)) {<br>
+ for (BasicBlock *SuccBB : successors(ClonedBB))<br>
+ SuccBB->removePredecessor(ClonedBB);<br>
+ DeadBlocks.push_back(ClonedBB);<br>
+ }<br>
+<br>
+ // Drop any remaining references to break cycles.<br>
+ for (BasicBlock *BB : DeadBlocks)<br>
+ BB->dropAllReferences();<br>
+ // Erase them from the IR.<br>
+ for (BasicBlock *BB : DeadBlocks)<br>
+ BB->eraseFromParent();<br>
+}<br>
+<br>
+static void<br>
deleteDeadBlocksFromLoop(Loop &L,<br>
- const SmallVectorImpl<BasicBlock *> &DeadBlocks,<br>
SmallVectorImpl<BasicBlock *> &ExitBlocks,<br>
DominatorTree &DT, LoopInfo &LI) {<br>
+ // Find all the dead blocks, and remove them from their successors.<br>
+ SmallVector<BasicBlock *, 16> DeadBlocks;<br>
+ for (BasicBlock *BB : llvm::concat<BasicBlock *const>(L.blocks(), ExitBlocks))<br>
+ if (!DT.isReachableFromEntry(BB)) {<br>
+ for (BasicBlock *SuccBB : successors(BB))<br>
+ SuccBB->removePredecessor(BB);<br>
+ DeadBlocks.push_back(BB);<br>
+ }<br>
+<br>
SmallPtrSet<BasicBlock *, 16> DeadBlockSet(DeadBlocks.begin(),<br>
DeadBlocks.end());<br>
<br>
@@ -1187,11 +1229,6 @@ deleteDeadBlocksFromLoop(Loop &L,<br>
llvm::erase_if(ExitBlocks,<br>
[&](BasicBlock *BB) { return DeadBlockSet.count(BB); });<br>
<br>
- // Remove these blocks from their successors.<br>
- for (auto *BB : DeadBlocks)<br>
- for (BasicBlock *SuccBB : successors(BB))<br>
- SuccBB->removePredecessor(BB, /*DontDeleteUselessPHIs*/ true);<br>
-<br>
// Walk from this loop up through its parents removing all of the dead blocks.<br>
for (Loop *ParentL = &L; ParentL; ParentL = ParentL->getParentLoop()) {<br>
for (auto *BB : DeadBlocks)<br>
@@ -1582,31 +1619,24 @@ void visitDomSubTree(DominatorTree &DT,<br>
} while (!DomWorklist.empty());<br>
}<br>
<br>
-/// Take an invariant branch that has been determined to be safe and worthwhile<br>
-/// to unswitch despite being non-trivial to do so and perform the unswitch.<br>
-///<br>
-/// This directly updates the CFG to hoist the predicate out of the loop, and<br>
-/// clone the necessary parts of the loop to maintain behavior.<br>
-///<br>
-/// It also updates both dominator tree and loopinfo based on the unswitching.<br>
-///<br>
-/// Once unswitching has been performed it runs the provided callback to report<br>
-/// the new loops and no-longer valid loops to the caller.<br>
-static bool unswitchInvariantBranch(<br>
- Loop &L, BranchInst &BI, ArrayRef<Value *> Invariants, DominatorTree &DT,<br>
- LoopInfo &LI, AssumptionCache &AC,<br>
+static bool unswitchNontrivialInvariants(<br>
+ Loop &L, TerminatorInst &TI, ArrayRef<Value *> Invariants,<br>
+ DominatorTree &DT, LoopInfo &LI, AssumptionCache &AC,<br>
function_ref<void(bool, ArrayRef<Loop *>)> UnswitchCB) {<br>
- auto *ParentBB = BI.getParent();<br>
-<br>
- // We can only unswitch conditional branches with an invariant condition or<br>
- // combining invariant conditions with an instruction.<br>
- assert(BI.isConditional() && "Can only unswitch a conditional branch!");<br>
- bool FullUnswitch = BI.getCondition() == Invariants[0];<br>
+ auto *ParentBB = TI.getParent();<br>
+ BranchInst *BI = dyn_cast<BranchInst>(&TI);<br>
+ SwitchInst *SI = BI ? nullptr : cast<SwitchInst>(&TI);<br>
+<br>
+ // We can only unswitch switches, conditional branches with an invariant<br>
+ // condition, or combining invariant conditions with an instruction.<br>
+ assert((SI || BI->isConditional()) &&<br>
+ "Can only unswitch switches and conditional branch!");<br>
+ bool FullUnswitch = SI || BI->getCondition() == Invariants[0];<br>
if (FullUnswitch)<br>
assert(Invariants.size() == 1 &&<br>
"Cannot have other invariants with full unswitching!");<br>
else<br>
- assert(isa<Instruction>(BI.getCondition()) &&<br>
+ assert(isa<Instruction>(BI->getCondition()) &&<br>
"Partial unswitching requires an instruction as the condition!");<br>
<br>
// Constant and BBs tracking the cloned and continuing successor. When we are<br>
@@ -1618,18 +1648,27 @@ static bool unswitchInvariantBranch(<br>
bool Direction = true;<br>
int ClonedSucc = 0;<br>
if (!FullUnswitch) {<br>
- if (cast<Instruction>(BI.getCondition())->getOpcode() != Instruction::Or) {<br>
- assert(cast<Instruction>(BI.getCondition())->getOpcode() == Instruction::And &&<br>
- "Only `or` and `and` instructions can combine invariants being unswitched.");<br>
+ if (cast<Instruction>(BI->getCondition())->getOpcode() != Instruction::Or) {<br>
+ assert(cast<Instruction>(BI->getCondition())->getOpcode() ==<br>
+ Instruction::And &&<br>
+ "Only `or` and `and` instructions can combine invariants being "<br>
+ "unswitched.");<br>
Direction = false;<br>
ClonedSucc = 1;<br>
}<br>
}<br>
- auto *UnswitchedSuccBB = BI.getSuccessor(ClonedSucc);<br>
- auto *ContinueSuccBB = BI.getSuccessor(1 - ClonedSucc);<br>
<br>
- assert(UnswitchedSuccBB != ContinueSuccBB &&<br>
- "Should not unswitch a branch that always goes to the same place!");<br>
+ BasicBlock *RetainedSuccBB =<br>
+ BI ? BI->getSuccessor(1 - ClonedSucc) : SI->getDefaultDest();<br>
+ SmallSetVector<BasicBlock *, 4> UnswitchedSuccBBs;<br>
+ if (BI)<br>
+ UnswitchedSuccBBs.insert(BI->getSuccessor(ClonedSucc));<br>
+ else<br>
+ for (auto Case : SI->cases())<br>
+ UnswitchedSuccBBs.insert(Case.getCaseSuccessor());<br>
+<br>
+ assert(!UnswitchedSuccBBs.count(RetainedSuccBB) &&<br>
+ "Should not unswitch the same successor we are retaining!");<br>
<br>
// The branch should be in this exact loop. Any inner loop's invariant branch<br>
// should be handled by unswitching that inner loop. The caller of this<br>
@@ -1648,9 +1687,6 @@ static bool unswitchInvariantBranch(<br>
if (isa<CleanupPadInst>(ExitBB->getFirstNonPHI()))<br>
return false;<br>
<br>
- SmallPtrSet<BasicBlock *, 4> ExitBlockSet(ExitBlocks.begin(),<br>
- ExitBlocks.end());<br>
-<br>
// Compute the parent loop now before we start hacking on things.<br>
Loop *ParentL = L.getParentLoop();<br>
<br>
@@ -1669,30 +1705,22 @@ static bool unswitchInvariantBranch(<br>
OuterExitL = NewOuterExitL;<br>
}<br>
<br>
- // If the edge we *aren't* cloning in the unswitch (the continuing edge)<br>
- // dominates its target, we can skip cloning the dominated region of the loop<br>
- // and its exits. We compute this as a set of nodes to be skipped.<br>
- SmallPtrSet<BasicBlock *, 4> SkippedLoopAndExitBlocks;<br>
- if (ContinueSuccBB->getUniquePredecessor() ||<br>
- llvm::all_of(predecessors(ContinueSuccBB), [&](BasicBlock *PredBB) {<br>
- return PredBB == ParentBB || DT.dominates(ContinueSuccBB, PredBB);<br>
- })) {<br>
- visitDomSubTree(DT, ContinueSuccBB, [&](BasicBlock *BB) {<br>
- SkippedLoopAndExitBlocks.insert(BB);<br>
- return true;<br>
- });<br>
- }<br>
- // If we are doing full unswitching, then similarly to the above, the edge we<br>
- // *are* cloning in the unswitch (the unswitched edge) dominates its target,<br>
- // we will end up with dead nodes in the original loop and its exits that will<br>
- // need to be deleted. Here, we just retain that the property holds and will<br>
- // compute the deleted set later.<br>
- bool DeleteUnswitchedSucc =<br>
- FullUnswitch &&<br>
- (UnswitchedSuccBB->getUniquePredecessor() ||<br>
- llvm::all_of(predecessors(UnswitchedSuccBB), [&](BasicBlock *PredBB) {<br>
- return PredBB == ParentBB || DT.dominates(UnswitchedSuccBB, PredBB);<br>
- }));<br>
+ // If the edge from this terminator to a successor dominates that successor,<br>
+ // store a map from each block in its dominator subtree to it. This lets us<br>
+ // tell when cloning for a particular successor if a block is dominated by<br>
+ // some *other* successor with a single data structure. We use this to<br>
+ // significantly reduce cloning.<br>
+ SmallDenseMap<BasicBlock *, BasicBlock *, 16> DominatingSucc;<br>
+ for (auto *SuccBB : llvm::concat<BasicBlock *const>(<br>
+ makeArrayRef(RetainedSuccBB), UnswitchedSuccBBs))<br>
+ if (SuccBB->getUniquePredecessor() ||<br>
+ llvm::all_of(predecessors(SuccBB), [&](BasicBlock *PredBB) {<br>
+ return PredBB == ParentBB || DT.dominates(SuccBB, PredBB);<br>
+ }))<br>
+ visitDomSubTree(DT, SuccBB, [&](BasicBlock *BB) {<br>
+ DominatingSucc[BB] = SuccBB;<br>
+ return true;<br>
+ });<br>
<br>
// Split the preheader, so that we know that there is a safe place to insert<br>
// the conditional branch. We will change the preheader to have a conditional<br>
@@ -1702,84 +1730,93 @@ static bool unswitchInvariantBranch(<br>
BasicBlock *SplitBB = L.getLoopPreheader();<br>
BasicBlock *LoopPH = SplitEdge(SplitBB, L.getHeader(), &DT, &LI);<br>
<br>
- // Keep a mapping for the cloned values.<br>
- ValueToValueMapTy VMap;<br>
-<br>
// Keep track of the dominator tree updates needed.<br>
SmallVector<DominatorTree::UpdateType, 4> DTUpdates;<br>
<br>
- // Build the cloned blocks from the loop.<br>
- auto *ClonedPH = buildClonedLoopBlocks(<br>
- L, LoopPH, SplitBB, ExitBlocks, ParentBB, UnswitchedSuccBB,<br>
- ContinueSuccBB, SkippedLoopAndExitBlocks, VMap, DTUpdates, AC, DT, LI);<br>
+ // Clone the loop for each unswitched successor.<br>
+ SmallVector<std::unique_ptr<ValueToValueMapTy>, 4> VMaps;<br>
+ VMaps.reserve(UnswitchedSuccBBs.size());<br>
+ SmallDenseMap<BasicBlock *, BasicBlock *, 4> ClonedPHs;<br>
+ for (auto *SuccBB : UnswitchedSuccBBs) {<br>
+ VMaps.emplace_back(new ValueToValueMapTy());<br>
+ ClonedPHs[SuccBB] = buildClonedLoopBlocks(<br>
+ L, LoopPH, SplitBB, ExitBlocks, ParentBB, SuccBB, RetainedSuccBB,<br>
+ DominatingSucc, *VMaps.back(), DTUpdates, AC, DT, LI);<br>
+ }<br>
<br>
// The stitching of the branched code back together depends on whether we're<br>
// doing full unswitching or not with the exception that we always want to<br>
// nuke the initial terminator placed in the split block.<br>
SplitBB->getTerminator()->eraseFromParent();<br>
if (FullUnswitch) {<br>
- // Remove the parent as a predecessor of the<br>
- // unswitched successor.<br>
- UnswitchedSuccBB->removePredecessor(ParentBB,<br>
- /*DontDeleteUselessPHIs*/ true);<br>
- DTUpdates.push_back({DominatorTree::Delete, ParentBB, UnswitchedSuccBB});<br>
-<br>
- // Now splice the branch from the original loop and use it to select between<br>
- // the two loops.<br>
- SplitBB->getInstList().splice(SplitBB->end(), ParentBB->getInstList(), BI);<br>
- BI.setSuccessor(ClonedSucc, ClonedPH);<br>
- BI.setSuccessor(1 - ClonedSucc, LoopPH);<br>
+ for (BasicBlock *SuccBB : UnswitchedSuccBBs) {<br>
+ // Remove the parent as a predecessor of the unswitched successor.<br>
+ SuccBB->removePredecessor(ParentBB,<br>
+ /*DontDeleteUselessPHIs*/ true);<br>
+ DTUpdates.push_back({DominatorTree::Delete, ParentBB, SuccBB});<br>
+ }<br>
+<br>
+ // Now splice the terminator from the original loop and rewrite its<br>
+ // successors.<br>
+ SplitBB->getInstList().splice(SplitBB->end(), ParentBB->getInstList(), TI);<br>
+ if (BI) {<br>
+ assert(UnswitchedSuccBBs.size() == 1 &&<br>
+ "Only one possible unswitched block for a branch!");<br>
+ BasicBlock *ClonedPH = ClonedPHs.begin()->second;<br>
+ BI->setSuccessor(ClonedSucc, ClonedPH);<br>
+ BI->setSuccessor(1 - ClonedSucc, LoopPH);<br>
+ DTUpdates.push_back({DominatorTree::Insert, SplitBB, ClonedPH});<br>
+ } else {<br>
+ assert(SI && "Must either be a branch or switch!");<br>
+<br>
+ // Walk the cases and directly update their successors.<br>
+ for (auto &Case : SI->cases())<br>
+ Case.setSuccessor(ClonedPHs.find(Case.getCaseSuccessor())->second);<br>
+ // We need to use the set to populate domtree updates as even when there<br>
+ // are multiple cases pointing at the same successor we only want to<br>
+ // insert one edge in the domtree.<br>
+ for (BasicBlock *SuccBB : UnswitchedSuccBBs)<br>
+ DTUpdates.push_back(<br>
+ {DominatorTree::Insert, SplitBB, ClonedPHs.find(SuccBB)->second});<br>
+<br>
+ SI->setDefaultDest(LoopPH);<br>
+ }<br>
<br>
// Create a new unconditional branch to the continuing block (as opposed to<br>
// the one cloned).<br>
- BranchInst::Create(ContinueSuccBB, ParentBB);<br>
+ BranchInst::Create(RetainedSuccBB, ParentBB);<br>
} else {<br>
+ assert(BI && "Only branches have partial unswitching.");<br>
+ assert(UnswitchedSuccBBs.size() == 1 &&<br>
+ "Only one possible unswitched block for a branch!");<br>
+ BasicBlock *ClonedPH = ClonedPHs.begin()->second;<br>
// When doing a partial unswitch, we have to do a bit more work to build up<br>
// the branch in the split block.<br>
buildPartialUnswitchConditionalBranch(*SplitBB, Invariants, Direction,<br>
*ClonedPH, *LoopPH);<br>
+ DTUpdates.push_back({DominatorTree::Insert, SplitBB, ClonedPH});<br>
}<br>
<br>
- // Before we update the dominator tree, collect the dead blocks if we're going<br>
- // to end up deleting the unswitched successor.<br>
- SmallVector<BasicBlock *, 16> DeadBlocks;<br>
- if (DeleteUnswitchedSucc) {<br>
- DeadBlocks.push_back(UnswitchedSuccBB);<br>
- for (int i = 0; i < (int)DeadBlocks.size(); ++i) {<br>
- // If we reach an exit block, stop recursing as the unswitched loop will<br>
- // end up reaching the merge block which we make the successor of the<br>
- // exit.<br>
- if (ExitBlockSet.count(DeadBlocks[i]))<br>
- continue;<br>
-<br>
- // Insert the children that are within the loop or exit block set. Other<br>
- // children may reach out of the loop. While we don't expect these to be<br>
- // dead (as the unswitched clone should reach them) we don't try to prove<br>
- // that here.<br>
- for (DomTreeNode *ChildN : *DT[DeadBlocks[i]])<br>
- if (L.contains(ChildN->getBlock()) ||<br>
- ExitBlockSet.count(ChildN->getBlock()))<br>
- DeadBlocks.push_back(ChildN->getBlock());<br>
- }<br>
- }<br>
-<br>
- // Add the remaining edge to our updates and apply them to get an up-to-date<br>
- // dominator tree. Note that this will cause the dead blocks above to be<br>
- // unreachable and no longer in the dominator tree.<br>
- DTUpdates.push_back({DominatorTree::Insert, SplitBB, ClonedPH});<br>
+ // Apply the updates accumulated above to get an up-to-date dominator tree.<br>
DT.applyUpdates(DTUpdates);<br>
<br>
+ // Now that we have an accurate dominator tree, first delete the dead cloned<br>
+ // blocks so that we can accurately build any cloned loops. It is important to<br>
+ // not delete the blocks from the original loop yet because we still want to<br>
+ // reference the original loop to understand the cloned loop's structure.<br>
+ deleteDeadClonedBlocks(L, ExitBlocks, VMaps, DT);<br>
+<br>
// Build the cloned loop structure itself. This may be substantially<br>
// different from the original structure due to the simplified CFG. This also<br>
// handles inserting all the cloned blocks into the correct loops.<br>
SmallVector<Loop *, 4> NonChildClonedLoops;<br>
- buildClonedLoops(L, ExitBlocks, VMap, LI, NonChildClonedLoops);<br>
-<br>
- // Delete anything that was made dead in the original loop due to<br>
- // unswitching.<br>
- if (!DeadBlocks.empty())<br>
- deleteDeadBlocksFromLoop(L, DeadBlocks, ExitBlocks, DT, LI);<br>
+ for (std::unique_ptr<ValueToValueMapTy> &VMap : VMaps)<br>
+ buildClonedLoops(L, ExitBlocks, *VMap, LI, NonChildClonedLoops);<br>
<br>
+ // Now that our cloned loops have been built, we can update the original loop.<br>
+ // First we delete the dead blocks from it and then we rebuild the loop<br>
+ // structure taking these deletions into account.<br>
+ deleteDeadBlocksFromLoop(L, ExitBlocks, DT, LI);<br>
SmallVector<Loop *, 4> HoistedLoops;<br>
bool IsStillLoop = rebuildLoopAfterUnswitch(L, ExitBlocks, LI, HoistedLoops);<br>
<br>
@@ -1790,31 +1827,37 @@ static bool unswitchInvariantBranch(<br>
// verification steps.<br>
assert(DT.verify(DominatorTree::VerificationLevel::Fast));<br>
<br>
- // Now we want to replace all the uses of the invariants within both the<br>
- // original and cloned blocks. We do this here so that we can use the now<br>
- // updated dominator tree to identify which side the users are on.<br>
- ConstantInt *UnswitchedReplacement =<br>
- Direction ? ConstantInt::getTrue(BI.getContext())<br>
- : ConstantInt::getFalse(BI.getContext());<br>
- ConstantInt *ContinueReplacement =<br>
- Direction ? ConstantInt::getFalse(BI.getContext())<br>
- : ConstantInt::getTrue(BI.getContext());<br>
- for (Value *Invariant : Invariants)<br>
- for (auto UI = Invariant->use_begin(), UE = Invariant->use_end();<br>
- UI != UE;) {<br>
- // Grab the use and walk past it so we can clobber it in the use list.<br>
- Use *U = &*UI++;<br>
- Instruction *UserI = dyn_cast<Instruction>(U->getUser());<br>
- if (!UserI)<br>
- continue;<br>
+ if (BI) {<br>
+ // If we unswitched a branch which collapses the condition to a known<br>
+ // constant we want to replace all the uses of the invariants within both<br>
+ // the original and cloned blocks. We do this here so that we can use the<br>
+ // now updated dominator tree to identify which side the users are on.<br>
+ assert(UnswitchedSuccBBs.size() == 1 &&<br>
+ "Only one possible unswitched block for a branch!");<br>
+ BasicBlock *ClonedPH = ClonedPHs.begin()->second;<br>
+ ConstantInt *UnswitchedReplacement =<br>
+ Direction ? ConstantInt::getTrue(BI->getContext())<br>
+ : ConstantInt::getFalse(BI->getContext());<br>
+ ConstantInt *ContinueReplacement =<br>
+ Direction ? ConstantInt::getFalse(BI->getContext())<br>
+ : ConstantInt::getTrue(BI->getContext());<br>
+ for (Value *Invariant : Invariants)<br>
+ for (auto UI = Invariant->use_begin(), UE = Invariant->use_end();<br>
+ UI != UE;) {<br>
+ // Grab the use and walk past it so we can clobber it in the use list.<br>
+ Use *U = &*UI++;<br>
+ Instruction *UserI = dyn_cast<Instruction>(U->getUser());<br>
+ if (!UserI)<br>
+ continue;<br>
<br>
- // Replace it with the 'continue' side if in the main loop body, and the<br>
- // unswitched if in the cloned blocks.<br>
- if (DT.dominates(LoopPH, UserI->getParent()))<br>
- U->set(ContinueReplacement);<br>
- else if (DT.dominates(ClonedPH, UserI->getParent()))<br>
- U->set(UnswitchedReplacement);<br>
- }<br>
+ // Replace it with the 'continue' side if in the main loop body, and the<br>
+ // unswitched if in the cloned blocks.<br>
+ if (DT.dominates(LoopPH, UserI->getParent()))<br>
+ U->set(ContinueReplacement);<br>
+ else if (DT.dominates(ClonedPH, UserI->getParent()))<br>
+ U->set(UnswitchedReplacement);<br>
+ }<br>
+ }<br>
<br>
// We can change which blocks are exit blocks of all the cloned sibling<br>
// loops, the current loop, and any parent loops which shared exit blocks<br>
@@ -1937,8 +1980,16 @@ static bool unswitchBestCondition(<br>
if (LI.getLoopFor(BB) != &L)<br>
continue;<br>
<br>
+ if (auto *SI = dyn_cast<SwitchInst>(BB->getTerminator())) {<br>
+ // We can only consider fully loop-invariant switch conditions as we need<br>
+ // to completely eliminate the switch after unswitching.<br>
+ if (!isa<Constant>(SI->getCondition()) &&<br>
+ L.isLoopInvariant(SI->getCondition()))<br>
+ UnswitchCandidates.push_back({SI, {SI->getCondition()}});<br>
+ continue;<br>
+ }<br>
+<br>
auto *BI = dyn_cast<BranchInst>(BB->getTerminator());<br>
- // FIXME: Handle switches here!<br>
if (!BI || !BI->isConditional() || isa<Constant>(BI->getCondition()) ||<br>
BI->getSuccessor(0) == BI->getSuccessor(1))<br>
continue;<br>
@@ -2091,9 +2142,9 @@ static bool unswitchBestCondition(<br>
TerminatorInst &TI = *TerminatorAndInvariants.first;<br>
ArrayRef<Value *> Invariants = TerminatorAndInvariants.second;<br>
BranchInst *BI = dyn_cast<BranchInst>(&TI);<br>
- int CandidateCost =<br>
- ComputeUnswitchedCost(TI, /*FullUnswitch*/ Invariants.size() == 1 && BI &&<br>
- Invariants[0] == BI->getCondition());<br>
+ int CandidateCost = ComputeUnswitchedCost(<br>
+ TI, /*FullUnswitch*/ !BI || (Invariants.size() == 1 &&<br>
+ Invariants[0] == BI->getCondition()));<br>
LLVM_DEBUG(dbgs() << " Computed cost of " << CandidateCost<br>
<< " for unswitch candidate: " << TI << "\n");<br>
if (!BestUnswitchTI || CandidateCost < BestUnswitchCost) {<br>
@@ -2109,17 +2160,11 @@ static bool unswitchBestCondition(<br>
return false;<br>
}<br>
<br>
- auto *UnswitchBI = dyn_cast<BranchInst>(BestUnswitchTI);<br>
- if (!UnswitchBI) {<br>
- // FIXME: Add support for unswitching a switch here!<br>
- LLVM_DEBUG(dbgs() << "Cannot unswitch anything but a branch!\n");<br>
- return false;<br>
- }<br>
-<br>
LLVM_DEBUG(dbgs() << " Trying to unswitch non-trivial (cost = "<br>
- << BestUnswitchCost << ") branch: " << *UnswitchBI << "\n");<br>
- return unswitchInvariantBranch(L, *UnswitchBI, BestUnswitchInvariants, DT, LI,<br>
- AC, UnswitchCB);<br>
+ << BestUnswitchCost << ") terminator: " << *BestUnswitchTI<br>
+ << "\n");<br>
+ return unswitchNontrivialInvariants(<br>
+ L, *BestUnswitchTI, BestUnswitchInvariants, DT, LI, AC, UnswitchCB);<br>
}<br>
<br>
/// Unswitch control flow predicated on loop invariant conditions.<br>
<br>
Modified: llvm/trunk/test/Transforms/SimpleLoopUnswitch/nontrivial-unswitch.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/SimpleLoopUnswitch/nontrivial-unswitch.ll?rev=335553&r1=335552&r2=335553&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/SimpleLoopUnswitch/nontrivial-unswitch.ll?rev=335553&r1=335552&r2=335553&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/Transforms/SimpleLoopUnswitch/nontrivial-unswitch.ll (original)<br>
+++ llvm/trunk/test/Transforms/SimpleLoopUnswitch/nontrivial-unswitch.ll Mon Jun 25 16:32:54 2018<br>
@@ -387,7 +387,7 @@ loop_begin:<br>
loop_b:<br>
%b = load i32, i32* %b.ptr<br>
br i1 %v, label %loop_begin, label %loop_exit<br>
-; The 'loop_b' unswitched loop.<br>
+; The original loop, now non-looping due to unswitching..<br>
;<br>
; CHECK: entry.split:<br>
; CHECK-NEXT: br label %loop_begin<br>
@@ -398,14 +398,13 @@ loop_b:<br>
; CHECK-NEXT: br label %loop_exit.split<br>
;<br>
; CHECK: loop_exit.split:<br>
-; CHECK-NEXT: %[[A_LCSSA:.*]] = phi i32 [ %[[A]], %loop_begin ]<br>
; CHECK-NEXT: br label %loop_exit<br>
<br>
loop_exit:<br>
%ab.phi = phi i32 [ %b, %loop_b ], [ %a, %loop_begin ]<br>
ret i32 %ab.phi<br>
; CHECK: loop_exit:<br>
-; CHECK-NEXT: %[[AB_PHI:.*]] = phi i32 [ %[[A_LCSSA]], %loop_exit.split ], [ %[[B_LCSSA]], %<a href="http://loop_exit.split.us" rel="noreferrer" target="_blank">loop_exit.split.us</a> ]<br>
+; CHECK-NEXT: %[[AB_PHI:.*]] = phi i32 [ %[[A]], %loop_exit.split ], [ %[[B_LCSSA]], %<a href="http://loop_exit.split.us" rel="noreferrer" target="_blank">loop_exit.split.us</a> ]<br>
; CHECK-NEXT: ret i32 %[[AB_PHI]]<br>
}<br>
<br>
@@ -458,8 +457,7 @@ loop_exit1:<br>
call void @sink1(i32 %a.phi)<br>
ret void<br>
; CHECK: loop_exit1:<br>
-; CHECK-NEXT: %[[A_PHI:.*]] = phi i32 [ %[[A_LCSSA]], %<a href="http://loop_exit1.split.us" rel="noreferrer" target="_blank">loop_exit1.split.us</a> ]<br>
-; CHECK-NEXT: call void @sink1(i32 %[[A_PHI]])<br>
+; CHECK-NEXT: call void @sink1(i32 %[[A_LCSSA]])<br>
; CHECK-NEXT: ret void<br>
<br>
loop_exit2:<br>
@@ -467,8 +465,8 @@ loop_exit2:<br>
call void @sink2(i32 %b.phi)<br>
ret void<br>
; CHECK: loop_exit2:<br>
-; CHECK-NEXT: %[[B_PHI:.*]] = phi i32 [ %[[B]], %loop_b ]<br>
-; CHECK-NEXT: call void @sink2(i32 %[[B_PHI]])<br>
+; CHECK-NEXT: %[[B_LCSSA:.*]] = phi i32 [ %[[B]], %loop_b ]<br>
+; CHECK-NEXT: call void @sink2(i32 %[[B_LCSSA]])<br>
; CHECK-NEXT: ret void<br>
}<br>
<br>
@@ -531,8 +529,7 @@ loop_exit2:<br>
call void @sink2(i32 %b.phi)<br>
ret void<br>
; CHECK: loop_exit2:<br>
-; CHECK-NEXT: %[[B_PHI:.*]] = phi i32 [ %[[B_LCSSA]], %<a href="http://loop_exit2.split.us" rel="noreferrer" target="_blank">loop_exit2.split.us</a> ]<br>
-; CHECK-NEXT: call void @sink2(i32 %[[B_PHI]])<br>
+; CHECK-NEXT: call void @sink2(i32 %[[B_LCSSA]])<br>
; CHECK-NEXT: ret void<br>
}<br>
<br>
@@ -587,8 +584,7 @@ loop_exit1:<br>
call void @sink1(i32 %a.phi)<br>
br label %exit<br>
; CHECK: loop_exit1:<br>
-; CHECK-NEXT: %[[A_PHI:.*]] = phi i32 [ %[[A_LCSSA]], %<a href="http://loop_exit1.split.us" rel="noreferrer" target="_blank">loop_exit1.split.us</a> ]<br>
-; CHECK-NEXT: call void @sink1(i32 %[[A_PHI]])<br>
+; CHECK-NEXT: call void @sink1(i32 %[[A_LCSSA]])<br>
; CHECK-NEXT: br label %exit<br>
<br>
loop_exit2:<br>
@@ -596,8 +592,8 @@ loop_exit2:<br>
call void @sink2(i32 %b.phi)<br>
br label %exit<br>
; CHECK: loop_exit2:<br>
-; CHECK-NEXT: %[[B_PHI:.*]] = phi i32 [ %[[B]], %loop_b ]<br>
-; CHECK-NEXT: call void @sink2(i32 %[[B_PHI]])<br>
+; CHECK-NEXT: %[[B_LCSSA:.*]] = phi i32 [ %[[B]], %loop_b ]<br>
+; CHECK-NEXT: call void @sink2(i32 %[[B_LCSSA]])<br>
; CHECK-NEXT: br label %exit<br>
<br>
exit:<br>
@@ -663,7 +659,7 @@ loop_latch:<br>
%v2 = load i1, i1* %ptr<br>
br i1 %v2, label %loop_begin, label %loop_exit<br>
; CHECK: loop_latch:<br>
-; CHECK-NEXT: %[[B_LCSSA:.*]] = phi i32 [ %[[B]], %inner_loop_b ]<br>
+; CHECK-NEXT: %[[B_INNER_LCSSA:.*]] = phi i32 [ %[[B]], %inner_loop_b ]<br>
; CHECK-NEXT: %[[V2:.*]] = load i1, i1* %ptr<br>
; CHECK-NEXT: br i1 %[[V2]], label %loop_begin, label %loop_exit.loopexit1<br>
<br>
@@ -671,15 +667,14 @@ loop_exit:<br>
%ab.phi = phi i32 [ %a, %inner_loop_begin ], [ %b.phi, %loop_latch ]<br>
ret i32 %ab.phi<br>
; CHECK: loop_exit.loopexit:<br>
-; CHECK-NEXT: %[[A_PHI:.*]] = phi i32 [ %[[A_LCSSA]], %<a href="http://loop_exit.loopexit.split.us" rel="noreferrer" target="_blank">loop_exit.loopexit.split.us</a> ]<br>
; CHECK-NEXT: br label %loop_exit<br>
;<br>
; CHECK: loop_exit.loopexit1:<br>
-; CHECK-NEXT: %[[B_PHI:.*]] = phi i32 [ %[[B_LCSSA]], %loop_latch ]<br>
+; CHECK-NEXT: %[[B_LCSSA:.*]] = phi i32 [ %[[B_INNER_LCSSA]], %loop_latch ]<br>
; CHECK-NEXT: br label %loop_exit<br>
;<br>
; CHECK: loop_exit:<br>
-; CHECK-NEXT: %[[AB_PHI:.*]] = phi i32 [ %[[A_PHI]], %loop_exit.loopexit ], [ %[[B_PHI]], %loop_exit.loopexit1 ]<br>
+; CHECK-NEXT: %[[AB_PHI:.*]] = phi i32 [ %[[A_LCSSA]], %loop_exit.loopexit ], [ %[[B_LCSSA]], %loop_exit.loopexit1 ]<br>
; CHECK-NEXT: ret i32 %[[AB_PHI]]<br>
}<br>
<br>
@@ -773,11 +768,10 @@ latch:<br>
; CHECK-NEXT: br label %latch<br>
;<br>
; CHECK: latch:<br>
-; CHECK-NEXT: %[[B_PHI:.*]] = phi i32 [ %[[B_INNER_LCSSA]], %loop_b_inner_exit ]<br>
; CHECK-NEXT: br i1 %[[V]], label %loop_begin, label %loop_exit.split<br>
;<br>
; CHECK: loop_exit.split:<br>
-; CHECK-NEXT: %[[B_LCSSA:.*]] = phi i32 [ %[[B_PHI]], %latch ]<br>
+; CHECK-NEXT: %[[B_LCSSA:.*]] = phi i32 [ %[[B_INNER_LCSSA]], %latch ]<br>
; CHECK-NEXT: br label %loop_exit<br>
<br>
loop_exit:<br>
@@ -1466,7 +1460,6 @@ inner_loop_exit:<br>
%v = load i1, i1* %ptr<br>
br i1 %v, label %loop_begin, label %loop_exit<br>
; CHECK: inner_loop_exit:<br>
-; CHECK-NEXT: %[[A_INNER_LCSSA:.*]] = phi i32 [ %[[A_INNER_LCSSA_US]], %<a href="http://inner_loop_exit.split.us" rel="noreferrer" target="_blank">inner_loop_exit.split.us</a> ]<br>
; CHECK-NEXT: %[[V:.*]] = load i1, i1* %ptr<br>
; CHECK-NEXT: br i1 %[[V]], label %loop_begin, label %loop_exit<br>
<br>
@@ -1474,7 +1467,7 @@ loop_exit:<br>
%a.lcssa = phi i32 [ %a.inner_lcssa, %inner_loop_exit ]<br>
ret i32 %a.lcssa<br>
; CHECK: loop_exit:<br>
-; CHECK-NEXT: %[[A_LCSSA:.*]] = phi i32 [ %[[A_INNER_LCSSA]], %inner_loop_exit ]<br>
+; CHECK-NEXT: %[[A_LCSSA:.*]] = phi i32 [ %[[A_INNER_LCSSA_US]], %inner_loop_exit ]<br>
; CHECK-NEXT: ret i32 %[[A_LCSSA]]<br>
}<br>
<br>
@@ -1555,7 +1548,7 @@ loop_exit:<br>
ret i32 %a.lcssa<br>
; CHECK: loop_exit:<br>
; CHECK-NEXT: %[[A_PHI:.*]] = phi i32 [ %[[A_LCSSA]], %loop_exit.split ], [ %[[A_PHI_US]], %<a href="http://loop_exit.split.us" rel="noreferrer" target="_blank">loop_exit.split.us</a> ]<br>
-; CHECK-NEXT: ret i32 %[[AB_PHI]]<br>
+; CHECK-NEXT: ret i32 %[[A_PHI]]<br>
}<br>
<br>
; Test that requires re-forming dedicated exits for the original loop.<br>
@@ -1635,7 +1628,7 @@ loop_exit:<br>
ret i32 %a.lcssa<br>
; CHECK: loop_exit:<br>
; CHECK-NEXT: %[[A_PHI:.*]] = phi i32 [ %[[A_PHI_SPLIT]], %loop_exit.split ], [ %[[A_LCSSA_US]], %<a href="http://loop_exit.split.us" rel="noreferrer" target="_blank">loop_exit.split.us</a> ]<br>
-; CHECK-NEXT: ret i32 %[[AB_PHI]]<br>
+; CHECK-NEXT: ret i32 %[[A_PHI]]<br>
}<br>
<br>
; Check that if a cloned inner loop after unswitching doesn't loop and directly<br>
@@ -1721,7 +1714,6 @@ loop_exit:<br>
%a.lcssa = phi i32 [ %a, %inner_loop_begin ], [ %a.inner_lcssa, %inner_loop_exit ]<br>
ret i32 %a.lcssa<br>
; CHECK: loop_exit.loopexit:<br>
-; CHECK-NEXT: %[[A_LCSSA_US:.*]] = phi i32 [ %[[A_INNER_LCSSA_US]], %<a href="http://loop_exit.loopexit.split.us" rel="noreferrer" target="_blank">loop_exit.loopexit.split.us</a> ]<br>
; CHECK-NEXT: br label %loop_exit<br>
;<br>
; CHECK: loop_exit.loopexit1:<br>
@@ -1729,7 +1721,7 @@ loop_exit:<br>
; CHECK-NEXT: br label %loop_exit<br>
;<br>
; CHECK: loop_exit:<br>
-; CHECK-NEXT: %[[A_PHI:.*]] = phi i32 [ %[[A_LCSSA_US]], %loop_exit.loopexit ], [ %[[A_LCSSA]], %loop_exit.loopexit1 ]<br>
+; CHECK-NEXT: %[[A_PHI:.*]] = phi i32 [ %[[A_INNER_LCSSA_US]], %loop_exit.loopexit ], [ %[[A_LCSSA]], %loop_exit.loopexit1 ]<br>
; CHECK-NEXT: ret i32 %[[A_PHI]]<br>
}<br>
<br>
@@ -1802,7 +1794,6 @@ inner_loop_exit:<br>
%v3 = load i1, i1* %ptr<br>
br i1 %v3, label %loop_latch, label %loop_exit<br>
; CHECK: inner_loop_exit:<br>
-; CHECK-NEXT: %[[A_INNER_PHI:.*]] = phi i32 [ %[[A_INNER_LCSSA_US]], %<a href="http://inner_loop_exit.split.us" rel="noreferrer" target="_blank">inner_loop_exit.split.us</a> ]<br>
; CHECK-NEXT: %[[V:.*]] = load i1, i1* %ptr<br>
; CHECK-NEXT: br i1 %[[V]], label %loop_latch, label %loop_exit.loopexit1<br>
<br>
@@ -1819,7 +1810,7 @@ loop_exit:<br>
; CHECK-NEXT: br label %loop_exit<br>
;<br>
; CHECK: loop_exit.loopexit1:<br>
-; CHECK-NEXT: %[[A_LCSSA_US:.*]] = phi i32 [ %[[A_INNER_PHI]], %inner_loop_exit ]<br>
+; CHECK-NEXT: %[[A_LCSSA_US:.*]] = phi i32 [ %[[A_INNER_LCSSA_US]], %inner_loop_exit ]<br>
; CHECK-NEXT: br label %loop_exit<br>
;<br>
; CHECK: loop_exit:<br>
@@ -1916,7 +1907,6 @@ inner_loop_exit:<br>
%v4 = load i1, i1* %ptr<br>
br i1 %v4, label %loop_begin, label %loop_exit<br>
; CHECK: inner_loop_exit.loopexit:<br>
-; CHECK-NEXT: %[[A_INNER_LCSSA_US:.*]] = phi i32 [ %[[A_INNER_INNER_LCSSA_US]], %<a href="http://inner_loop_exit.loopexit.split.us" rel="noreferrer" target="_blank">inner_loop_exit.loopexit.split.us</a> ]<br>
; CHECK-NEXT: br label %inner_loop_exit<br>
;<br>
; CHECK: inner_loop_exit.loopexit1:<br>
@@ -1924,7 +1914,7 @@ inner_loop_exit:<br>
; CHECK-NEXT: br label %inner_loop_exit<br>
;<br>
; CHECK: inner_loop_exit:<br>
-; CHECK-NEXT: %[[A_INNER_PHI:.*]] = phi i32 [ %[[A_INNER_LCSSA_US]], %inner_loop_exit.loopexit ], [ %[[A_INNER_LCSSA]], %inner_loop_exit.loopexit1 ]<br>
+; CHECK-NEXT: %[[A_INNER_PHI:.*]] = phi i32 [ %[[A_INNER_INNER_LCSSA_US]], %inner_loop_exit.loopexit ], [ %[[A_INNER_LCSSA]], %inner_loop_exit.loopexit1 ]<br>
; CHECK-NEXT: %[[V:.*]] = load i1, i1* %ptr<br>
; CHECK-NEXT: br i1 %[[V]], label %loop_begin, label %loop_exit<br>
<br>
@@ -2010,7 +2000,6 @@ inner_inner_loop_exit:<br>
%v3 = load i1, i1* %ptr<br>
br i1 %v3, label %inner_loop_latch, label %inner_loop_exit<br>
; CHECK: inner_inner_loop_exit:<br>
-; CHECK-NEXT: %[[A_INNER_INNER_PHI:.*]] = phi i32 [ %[[A_INNER_INNER_LCSSA_US]], %<a href="http://inner_inner_loop_exit.split.us" rel="noreferrer" target="_blank">inner_inner_loop_exit.split.us</a> ]<br>
; CHECK-NEXT: %[[V:.*]] = load i1, i1* %ptr<br>
; CHECK-NEXT: br i1 %[[V]], label %inner_loop_latch, label %inner_loop_exit.loopexit1<br>
<br>
@@ -2028,7 +2017,7 @@ inner_loop_exit:<br>
; CHECK-NEXT: br label %inner_loop_exit<br>
;<br>
; CHECK: inner_loop_exit.loopexit1:<br>
-; CHECK-NEXT: %[[A_INNER_LCSSA_US:.*]] = phi i32 [ %[[A_INNER_INNER_PHI]], %inner_inner_loop_exit ]<br>
+; CHECK-NEXT: %[[A_INNER_LCSSA_US:.*]] = phi i32 [ %[[A_INNER_INNER_LCSSA_US]], %inner_inner_loop_exit ]<br>
; CHECK-NEXT: br label %inner_loop_exit<br>
;<br>
; CHECK: inner_loop_exit:<br>
@@ -2296,56 +2285,96 @@ define i32 @test20(i32* %var, i32 %cond1<br>
entry:<br>
br label %loop_begin<br>
; CHECK-NEXT: entry:<br>
-; CHECK-NEXT: br label %loop_begin<br>
+; CHECK-NEXT: switch i32 %cond2, label %[[ENTRY_SPLIT_EXIT:.*]] [<br>
+; CHECK-NEXT: i32 0, label %[[ENTRY_SPLIT_A:.*]]<br>
+; CHECK-NEXT: i32 1, label %[[ENTRY_SPLIT_A]]<br>
+; CHECK-NEXT: i32 13, label %[[ENTRY_SPLIT_B:.*]]<br>
+; CHECK-NEXT: i32 2, label %[[ENTRY_SPLIT_A]]<br>
+; CHECK-NEXT: i32 42, label %[[ENTRY_SPLIT_C:.*]]<br>
+; CHECK-NEXT: ]<br>
<br>
loop_begin:<br>
%var_val = load i32, i32* %var<br>
- switch i32 %cond2, label %loop_a [<br>
- i32 0, label %loop_b<br>
- i32 1, label %loop_b<br>
- i32 13, label %loop_c<br>
- i32 2, label %loop_b<br>
- i32 42, label %loop_exit<br>
+ switch i32 %cond2, label %loop_exit [<br>
+ i32 0, label %loop_a<br>
+ i32 1, label %loop_a<br>
+ i32 13, label %loop_b<br>
+ i32 2, label %loop_a<br>
+ i32 42, label %loop_c<br>
]<br>
-; CHECK: loop_begin:<br>
-; CHECK-NEXT: %[[V:.*]] = load i32, i32* %var<br>
-; CHECK-NEXT: switch i32 %cond2, label %loop_a [<br>
-; CHECK-NEXT: i32 0, label %loop_b<br>
-; CHECK-NEXT: i32 1, label %loop_b<br>
-; CHECK-NEXT: i32 13, label %loop_c<br>
-; CHECK-NEXT: i32 2, label %loop_b<br>
-; CHECK-NEXT: i32 42, label %loop_exit<br>
-; CHECK-NEXT: ]<br>
<br>
loop_a:<br>
call void @a()<br>
br label %loop_latch<br>
-; CHECK: loop_a:<br>
+; Unswitched 'a' loop.<br>
+;<br>
+; CHECK: [[ENTRY_SPLIT_A]]:<br>
+; CHECK-NEXT: br label %[[LOOP_BEGIN_A:.*]]<br>
+;<br>
+; CHECK: [[LOOP_BEGIN_A]]:<br>
+; CHECK-NEXT: %{{.*}} = load i32, i32* %var<br>
+; CHECK-NEXT: br label %[[LOOP_A:.*]]<br>
+;<br>
+; CHECK: [[LOOP_A]]:<br>
; CHECK-NEXT: call void @a()<br>
-; CHECK-NEXT: br label %loop_latch<br>
+; CHECK-NEXT: br label %[[LOOP_LATCH_A:.*]]<br>
+;<br>
+; CHECK: [[LOOP_LATCH_A]]:<br>
+; CHECK: br label %[[LOOP_BEGIN_A]]<br>
<br>
loop_b:<br>
call void @b()<br>
br label %loop_latch<br>
-; CHECK: loop_b:<br>
+; Unswitched 'b' loop.<br>
+;<br>
+; CHECK: [[ENTRY_SPLIT_B]]:<br>
+; CHECK-NEXT: br label %[[LOOP_BEGIN_B:.*]]<br>
+;<br>
+; CHECK: [[LOOP_BEGIN_B]]:<br>
+; CHECK-NEXT: %{{.*}} = load i32, i32* %var<br>
+; CHECK-NEXT: br label %[[LOOP_B:.*]]<br>
+;<br>
+; CHECK: [[LOOP_B]]:<br>
; CHECK-NEXT: call void @b()<br>
-; CHECK-NEXT: br label %loop_latch<br>
+; CHECK-NEXT: br label %[[LOOP_LATCH_B:.*]]<br>
+;<br>
+; CHECK: [[LOOP_LATCH_B]]:<br>
+; CHECK: br label %[[LOOP_BEGIN_B]]<br>
<br>
loop_c:<br>
call void @c() noreturn nounwind<br>
br label %loop_latch<br>
-; CHECK: loop_c:<br>
+; Unswitched 'c' loop.<br>
+;<br>
+; CHECK: [[ENTRY_SPLIT_C]]:<br>
+; CHECK-NEXT: br label %[[LOOP_BEGIN_C:.*]]<br>
+;<br>
+; CHECK: [[LOOP_BEGIN_C]]:<br>
+; CHECK-NEXT: %{{.*}} = load i32, i32* %var<br>
+; CHECK-NEXT: br label %[[LOOP_C:.*]]<br>
+;<br>
+; CHECK: [[LOOP_C]]:<br>
; CHECK-NEXT: call void @c()<br>
-; CHECK-NEXT: br label %loop_latch<br>
+; CHECK-NEXT: br label %[[LOOP_LATCH_C:.*]]<br>
+;<br>
+; CHECK: [[LOOP_LATCH_C]]:<br>
+; CHECK: br label %[[LOOP_BEGIN_C]]<br>
<br>
loop_latch:<br>
br label %loop_begin<br>
-; CHECK: loop_latch:<br>
-; CHECK-NEXT: br label %loop_begin<br>
<br>
loop_exit:<br>
%lcssa = phi i32 [ %var_val, %loop_begin ]<br>
ret i32 %lcssa<br>
+; Unswitched exit edge (no longer a loop).<br>
+;<br>
+; CHECK: [[ENTRY_SPLIT_EXIT]]:<br>
+; CHECK-NEXT: br label %loop_begin<br>
+;<br>
+; CHECK: loop_begin:<br>
+; CHECK-NEXT: %[[V:.*]] = load i32, i32* %var<br>
+; CHECK-NEXT: br label %loop_exit<br>
+;<br>
; CHECK: loop_exit:<br>
; CHECK-NEXT: %[[LCSSA:.*]] = phi i32 [ %[[V]], %loop_begin ]<br>
; CHECK-NEXT: ret i32 %[[LCSSA]]<br>
@@ -2824,3 +2853,112 @@ loop_exit:<br>
; CHECK: loop_exit:<br>
; CHECK-NEXT: ret<br>
}<br>
+<br>
+; Non-trivial unswitching of a switch.<br>
+define i32 @test27(i1* %ptr, i32 %cond) {<br>
+; CHECK-LABEL: @test27(<br>
+entry:<br>
+ br label %loop_begin<br>
+; CHECK-NEXT: entry:<br>
+; CHECK-NEXT: switch i32 %cond, label %[[ENTRY_SPLIT_LATCH:.*]] [<br>
+; CHECK-NEXT: i32 0, label %[[ENTRY_SPLIT_A:.*]]<br>
+; CHECK-NEXT: i32 1, label %[[ENTRY_SPLIT_B:.*]]<br>
+; CHECK-NEXT: i32 2, label %[[ENTRY_SPLIT_C:.*]]<br>
+; CHECK-NEXT: ]<br>
+<br>
+loop_begin:<br>
+ switch i32 %cond, label %latch [<br>
+ i32 0, label %loop_a<br>
+ i32 1, label %loop_b<br>
+ i32 2, label %loop_c<br>
+ ]<br>
+<br>
+loop_a:<br>
+ call void @a()<br>
+ br label %latch<br>
+; Unswitched 'a' loop.<br>
+;<br>
+; CHECK: [[ENTRY_SPLIT_A]]:<br>
+; CHECK-NEXT: br label %[[LOOP_BEGIN_A:.*]]<br>
+;<br>
+; CHECK: [[LOOP_BEGIN_A]]:<br>
+; CHECK-NEXT: br label %[[LOOP_A:.*]]<br>
+;<br>
+; CHECK: [[LOOP_A]]:<br>
+; CHECK-NEXT: call void @a()<br>
+; CHECK-NEXT: br label %[[LOOP_LATCH_A:.*]]<br>
+;<br>
+; CHECK: [[LOOP_LATCH_A]]:<br>
+; CHECK-NEXT: %[[V_A:.*]] = load i1, i1* %ptr<br>
+; CHECK: br i1 %[[V_A]], label %[[LOOP_BEGIN_A]], label %[[LOOP_EXIT_A:.*]]<br>
+;<br>
+; CHECK: [[LOOP_EXIT_A]]:<br>
+; CHECK-NEXT: br label %loop_exit<br>
+<br>
+loop_b:<br>
+ call void @b()<br>
+ br label %latch<br>
+; Unswitched 'b' loop.<br>
+;<br>
+; CHECK: [[ENTRY_SPLIT_B]]:<br>
+; CHECK-NEXT: br label %[[LOOP_BEGIN_B:.*]]<br>
+;<br>
+; CHECK: [[LOOP_BEGIN_B]]:<br>
+; CHECK-NEXT: br label %[[LOOP_B:.*]]<br>
+;<br>
+; CHECK: [[LOOP_B]]:<br>
+; CHECK-NEXT: call void @b()<br>
+; CHECK-NEXT: br label %[[LOOP_LATCH_B:.*]]<br>
+;<br>
+; CHECK: [[LOOP_LATCH_B]]:<br>
+; CHECK-NEXT: %[[V_B:.*]] = load i1, i1* %ptr<br>
+; CHECK: br i1 %[[V_B]], label %[[LOOP_BEGIN_B]], label %[[LOOP_EXIT_B:.*]]<br>
+;<br>
+; CHECK: [[LOOP_EXIT_B]]:<br>
+; CHECK-NEXT: br label %loop_exit<br>
+<br>
+loop_c:<br>
+ call void @c()<br>
+ br label %latch<br>
+; Unswitched 'c' loop.<br>
+;<br>
+; CHECK: [[ENTRY_SPLIT_C]]:<br>
+; CHECK-NEXT: br label %[[LOOP_BEGIN_C:.*]]<br>
+;<br>
+; CHECK: [[LOOP_BEGIN_C]]:<br>
+; CHECK-NEXT: br label %[[LOOP_C:.*]]<br>
+;<br>
+; CHECK: [[LOOP_C]]:<br>
+; CHECK-NEXT: call void @c()<br>
+; CHECK-NEXT: br label %[[LOOP_LATCH_C:.*]]<br>
+;<br>
+; CHECK: [[LOOP_LATCH_C]]:<br>
+; CHECK-NEXT: %[[V_C:.*]] = load i1, i1* %ptr<br>
+; CHECK: br i1 %[[V_C]], label %[[LOOP_BEGIN_C]], label %[[LOOP_EXIT_C:.*]]<br>
+;<br>
+; CHECK: [[LOOP_EXIT_C]]:<br>
+; CHECK-NEXT: br label %loop_exit<br>
+<br>
+latch:<br>
+ %v = load i1, i1* %ptr<br>
+ br i1 %v, label %loop_begin, label %loop_exit<br>
+; Unswitched the 'latch' only loop.<br>
+;<br>
+; CHECK: [[ENTRY_SPLIT_LATCH]]:<br>
+; CHECK-NEXT: br label %[[LOOP_BEGIN_LATCH:.*]]<br>
+;<br>
+; CHECK: [[LOOP_BEGIN_LATCH]]:<br>
+; CHECK-NEXT: br label %[[LOOP_LATCH_LATCH:.*]]<br>
+;<br>
+; CHECK: [[LOOP_LATCH_LATCH]]:<br>
+; CHECK-NEXT: %[[V_LATCH:.*]] = load i1, i1* %ptr<br>
+; CHECK: br i1 %[[V_LATCH]], label %[[LOOP_BEGIN_LATCH]], label %[[LOOP_EXIT_LATCH:.*]]<br>
+;<br>
+; CHECK: [[LOOP_EXIT_LATCH]]:<br>
+; CHECK-NEXT: br label %loop_exit<br>
+<br>
+loop_exit:<br>
+ ret i32 0<br>
+; CHECK: loop_exit:<br>
+; CHECK-NEXT: ret i32 0<br>
+}<br>
\ No newline at end of file<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div>
</blockquote></div>
</blockquote></div>