<div dir="ltr">Hi Kyle, I'd like to do a post-commit review of the two bug fixes on top of the original patch. Can you create a diff in phabricator?<div><br></div><div>thanks,</div><div><br>David</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Oct 10, 2016 at 6:20 PM, Kyle Butt via llvm-commits <span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: iteratee<br>
Date: Mon Oct 10 20:20:33 2016<br>
New Revision: 283842<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=283842&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project?rev=283842&view=rev</a><br>
Log:<br>
Codegen: Tail-duplicate during placement.<br>
<br>
The tail duplication pass uses an assumed layout when making duplication<br>
decisions. This is fine, but passes up duplication opportunities that<br>
may arise when blocks are outlined. Because we want the updated CFG to<br>
affect subsequent placement decisions, this change must occur during<br>
placement.<br>
<br>
In order to achieve this goal, TailDuplicationPass is split into a<br>
utility class, TailDuplicator, and the pass itself. The pass delegates<br>
nearly everything to the TailDuplicator object, except for looping over<br>
the blocks in a function. This allows the same code to be used for tail<br>
duplication in both places.<br>
<br>
This change, in concert with outlining optional branches, allows<br>
triangle shaped code to perform much better, esepecially when the<br>
taken/untaken branches are correlated, as it creates a second spine when<br>
the tests are small enough.<br>
<br>
Issue from previous rollback fixed, and a new test was added for that<br>
case as well. Issue was worklist/scheduling/taildup issue in layout.<br>
<br>
Issue from 2nd rollback fixed, with 2 additional tests. Issue was<br>
tail merging/loop info/tail-duplication causing issue with loops that share<br>
a header block.<br>
<br>
Issue with early tail-duplication of blocks that branch to a fallthrough<br>
predecessor fixed with test case: tail-dup-branch-to-<wbr>fallthrough.ll<br>
<br>
Differential revision: <a href="https://reviews.llvm.org/D18226" rel="noreferrer" target="_blank">https://reviews.llvm.org/<wbr>D18226</a><br>
<br>
Added:<br>
llvm/trunk/test/CodeGen/<wbr>AArch64/tail-dup-repeat-<wbr>worklist.ll<br>
llvm/trunk/test/CodeGen/<wbr>PowerPC/tail-dup-branch-to-<wbr>fallthrough.ll<br>
llvm/trunk/test/CodeGen/<wbr>PowerPC/tail-dup-layout.ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>tail-dup-merge-loop-headers.ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>tail-dup-repeat.ll<br>
Modified:<br>
llvm/trunk/include/llvm/<wbr>Analysis/LoopInfoImpl.h<br>
llvm/trunk/include/llvm/<wbr>CodeGen/TailDuplicator.h<br>
llvm/trunk/lib/CodeGen/<wbr>MachineBlockPlacement.cpp<br>
llvm/trunk/lib/CodeGen/<wbr>TailDuplication.cpp<br>
llvm/trunk/lib/CodeGen/<wbr>TailDuplicator.cpp<br>
llvm/trunk/test/CodeGen/<wbr>AArch64/arm64-extload-<wbr>knownzero.ll<br>
llvm/trunk/test/CodeGen/<wbr>AArch64/machine_cse.ll<br>
llvm/trunk/test/CodeGen/ARM/<wbr>2011-03-23-PeepholeBug.ll<br>
llvm/trunk/test/CodeGen/<wbr>PowerPC/branch-opt.ll<br>
llvm/trunk/test/CodeGen/<wbr>PowerPC/sjlj.ll<br>
llvm/trunk/test/CodeGen/<wbr>WebAssembly/cfg-stackify.ll<br>
llvm/trunk/test/CodeGen/<wbr>WebAssembly/mem-intrinsics.ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>block-placement.ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>cmov-into-branch.ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>fma-intrinsics-phi-213-to-231.<wbr>ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>fp-une-cmp.ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>pr11202.ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>ragreedy-bug.ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>sse1.ll<br>
llvm/trunk/test/CodeGen/X86/<wbr>update-terminator.mir<br>
<br>
Modified: llvm/trunk/include/llvm/<wbr>Analysis/LoopInfoImpl.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Analysis/LoopInfoImpl.h?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/include/<wbr>llvm/Analysis/LoopInfoImpl.h?<wbr>rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/include/llvm/<wbr>Analysis/LoopInfoImpl.h (original)<br>
+++ llvm/trunk/include/llvm/<wbr>Analysis/LoopInfoImpl.h Mon Oct 10 20:20:33 2016<br>
@@ -186,8 +186,13 @@ BlockT *LoopBase<BlockT, LoopT>::getLoop<br>
template<class BlockT, class LoopT><br>
void LoopBase<BlockT, LoopT>::<br>
addBasicBlockToLoop(BlockT *NewBB, LoopInfoBase<BlockT, LoopT> &LIB) {<br>
- assert((Blocks.empty() || LIB[getHeader()] == this) &&<br>
- "Incorrect LI specified for this loop!");<br>
+#ifndef NDEBUG<br>
+ if (!Blocks.empty()) {<br>
+ auto SameHeader = LIB[getHeader()];<br>
+ assert(contains(SameHeader) && getHeader() == SameHeader->getHeader()<br>
+ && "Incorrect LI specified for this loop!");<br>
+ }<br>
+#endif<br>
assert(NewBB && "Cannot add a null basic block to the loop!");<br>
assert(!LIB[NewBB] && "BasicBlock already in the loop!");<br>
<br>
<br>
Modified: llvm/trunk/include/llvm/<wbr>CodeGen/TailDuplicator.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/TailDuplicator.h?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/include/<wbr>llvm/CodeGen/TailDuplicator.h?<wbr>rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/include/llvm/<wbr>CodeGen/TailDuplicator.h (original)<br>
+++ llvm/trunk/include/llvm/<wbr>CodeGen/TailDuplicator.h Mon Oct 10 20:20:33 2016<br>
@@ -15,6 +15,7 @@<br>
#ifndef LLVM_CODEGEN_TAILDUPLICATOR_H<br>
#define LLVM_CODEGEN_TAILDUPLICATOR_H<br>
<br>
+#include "llvm/ADT/STLExtras.h"<br>
#include "llvm/CodeGen/<wbr>MachineBranchProbabilityInfo.<wbr>h"<br>
#include "llvm/CodeGen/<wbr>MachineModuleInfo.h"<br>
#include "llvm/CodeGen/<wbr>MachineRegisterInfo.h"<br>
@@ -37,6 +38,7 @@ class TailDuplicator {<br>
MachineRegisterInfo *MRI;<br>
MachineFunction *MF;<br>
bool PreRegAlloc;<br>
+ bool LayoutMode;<br>
unsigned TailDupSize;<br>
<br>
// A list of virtual registers for which to update SSA form.<br>
@@ -50,10 +52,16 @@ class TailDuplicator {<br>
<br>
public:<br>
/// Prepare to run on a specific machine function.<br>
- /// @param TailDupSize - Maxmimum size of blocks to tail-duplicate.<br>
+ /// @param MF - Function that will be processed<br>
+ /// @param MBPI - Branch Probability Info. Used to propagate correct<br>
+ /// probabilities when modifying the CFG.<br>
+ /// @param LayoutMode - When true, don't use the existing layout to make<br>
+ /// decisions.<br>
+ /// @param TailDupSize - Maxmimum size of blocks to tail-duplicate. Zero<br>
+ /// default implies using the command line value TailDupSize.<br>
void initMF(MachineFunction &MF,<br>
const MachineBranchProbabilityInfo *MBPI,<br>
- unsigned TailDupSize = 0);<br>
+ bool LayoutMode, unsigned TailDupSize = 0);<br>
bool tailDuplicateBlocks();<br>
static bool isSimpleBB(MachineBasicBlock *TailBB);<br>
bool shouldTailDuplicate(bool IsSimple, MachineBasicBlock &TailBB);<br>
@@ -63,9 +71,13 @@ public:<br>
/// up.<br>
/// If \p DuplicatePreds is not null, it will be updated to contain the list<br>
/// of predecessors that received a copy of \p MBB.<br>
+ /// If \p RemovalCallback is non-null. It will be called before MBB is<br>
+ /// deleted.<br>
bool tailDuplicateAndUpdate(<br>
bool IsSimple, MachineBasicBlock *MBB,<br>
- SmallVectorImpl<<wbr>MachineBasicBlock*> *DuplicatedPreds = nullptr);<br>
+ MachineBasicBlock *ForcedLayoutPred,<br>
+ SmallVectorImpl<<wbr>MachineBasicBlock*> *DuplicatedPreds = nullptr,<br>
+ llvm::function_ref<void(<wbr>MachineBasicBlock *)> *RemovalCallback = nullptr);<br>
<br>
private:<br>
typedef TargetInstrInfo::RegSubRegPair RegSubRegPair;<br>
@@ -89,14 +101,18 @@ private:<br>
SmallVectorImpl<<wbr>MachineBasicBlock *> &TDBBs,<br>
const DenseSet<unsigned> &RegsUsedByPhi,<br>
SmallVectorImpl<MachineInstr *> &Copies);<br>
- bool tailDuplicate(bool IsSimple, MachineBasicBlock *TailBB,<br>
+ bool tailDuplicate(bool IsSimple,<br>
+ MachineBasicBlock *TailBB,<br>
+ MachineBasicBlock *ForcedLayoutPred,<br>
SmallVectorImpl<<wbr>MachineBasicBlock *> &TDBBs,<br>
SmallVectorImpl<MachineInstr *> &Copies);<br>
void appendCopies(MachineBasicBlock *MBB,<br>
SmallVectorImpl<std::pair<<wbr>unsigned,RegSubRegPair>> &CopyInfos,<br>
SmallVectorImpl<MachineInstr *> &Copies);<br>
<br>
- void removeDeadBlock(<wbr>MachineBasicBlock *MBB);<br>
+ void removeDeadBlock(<br>
+ MachineBasicBlock *MBB,<br>
+ llvm::function_ref<void(<wbr>MachineBasicBlock *)> *RemovalCallback = nullptr);<br>
};<br>
<br>
} // End llvm namespace<br>
<br>
Modified: llvm/trunk/lib/CodeGen/<wbr>MachineBlockPlacement.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/MachineBlockPlacement.cpp?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/<wbr>CodeGen/MachineBlockPlacement.<wbr>cpp?rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/lib/CodeGen/<wbr>MachineBlockPlacement.cpp (original)<br>
+++ llvm/trunk/lib/CodeGen/<wbr>MachineBlockPlacement.cpp Mon Oct 10 20:20:33 2016<br>
@@ -40,6 +40,7 @@<br>
#include "llvm/CodeGen/<wbr>MachineFunctionPass.h"<br>
#include "llvm/CodeGen/MachineLoopInfo.<wbr>h"<br>
#include "llvm/CodeGen/<wbr>MachineModuleInfo.h"<br>
+#include "llvm/CodeGen/TailDuplicator.<wbr>h"<br>
#include "llvm/Support/Allocator.h"<br>
#include "llvm/Support/CommandLine.h"<br>
#include "llvm/Support/Debug.h"<br>
@@ -121,6 +122,12 @@ static cl::opt<unsigned> MisfetchCost(<br>
static cl::opt<unsigned> JumpInstCost("jump-inst-cost",<br>
cl::desc("Cost of jump instructions."),<br>
cl::init(1), cl::Hidden);<br>
+static cl::opt<bool><br>
+TailDupPlacement("tail-dup-<wbr>placement",<br>
+ cl::desc("Perform tail duplication during placement. "<br>
+ "Creates more fallthrough opportunites in "<br>
+ "outline branches."),<br>
+ cl::init(true), cl::Hidden);<br>
<br>
static cl::opt<bool><br>
BranchFoldPlacement("branch-<wbr>fold-placement",<br>
@@ -128,6 +135,14 @@ BranchFoldPlacement("branch-<wbr>fold-placeme<br>
"Reduces code size."),<br>
cl::init(true), cl::Hidden);<br>
<br>
+// Heuristic for tail duplication.<br>
+static cl::opt<unsigned> TailDuplicatePlacementThreshol<wbr>d(<br>
+ "tail-dup-placement-threshold"<wbr>,<br>
+ cl::desc("Instruction cutoff for tail duplication during layout. "<br>
+ "Tail merging during layout is forced to have a threshold "<br>
+ "that won't conflict."), cl::init(2),<br>
+ cl::Hidden);<br>
+<br>
extern cl::opt<unsigned> StaticLikelyProb;<br>
extern cl::opt<unsigned> ProfileLikelyProb;<br>
<br>
@@ -185,6 +200,16 @@ public:<br>
/// \brief End of blocks within the chain.<br>
iterator end() { return Blocks.end(); }<br>
<br>
+ bool remove(MachineBasicBlock* BB) {<br>
+ for(iterator i = begin(); i != end(); ++i) {<br>
+ if (*i == BB) {<br>
+ Blocks.erase(i);<br>
+ return true;<br>
+ }<br>
+ }<br>
+ return false;<br>
+ }<br>
+<br>
/// \brief Merge a block chain into this one.<br>
///<br>
/// This routine merges a block chain into this one. It takes care of forming<br>
@@ -266,6 +291,13 @@ class MachineBlockPlacement : public Mac<br>
/// \brief A handle to the post dominator tree.<br>
MachineDominatorTree *MDT;<br>
<br>
+ /// \brief Duplicator used to duplicate tails during placement.<br>
+ ///<br>
+ /// Placement decisions can open up new tail duplication opportunities, but<br>
+ /// since tail duplication affects placement decisions of later blocks, it<br>
+ /// must be done inline.<br>
+ TailDuplicator TailDup;<br>
+<br>
/// \brief A set of blocks that are unavoidably execute, i.e. they dominate<br>
/// all terminators of the MachineFunction.<br>
SmallPtrSet<MachineBasicBlock *, 4> UnavoidableBlocks;<br>
@@ -287,8 +319,18 @@ class MachineBlockPlacement : public Mac<br>
/// between basic blocks.<br>
DenseMap<MachineBasicBlock *, BlockChain *> BlockToChain;<br>
<br>
+ /// Decrease the UnscheduledPredecessors count for all blocks in chain, and<br>
+ /// if the count goes to 0, add them to the appropriate work list.<br>
void markChainSuccessors(BlockChain &Chain, MachineBasicBlock *LoopHeaderBB,<br>
const BlockFilterSet *BlockFilter = nullptr);<br>
+<br>
+ /// Decrease the UnscheduledPredecessors count for a single block, and<br>
+ /// if the count goes to 0, add them to the appropriate work list.<br>
+ void markBlockSuccessors(<br>
+ BlockChain &Chain, MachineBasicBlock *BB, MachineBasicBlock *LoopHeaderBB,<br>
+ const BlockFilterSet *BlockFilter = nullptr);<br>
+<br>
+<br>
BranchProbability<br>
collectViableSuccessors(<wbr>MachineBasicBlock *BB, BlockChain &Chain,<br>
const BlockFilterSet *BlockFilter,<br>
@@ -298,6 +340,16 @@ class MachineBlockPlacement : public Mac<br>
const BlockFilterSet *BlockFilter,<br>
BranchProbability SuccProb,<br>
BranchProbability HotProb);<br>
+ bool repeatedlyTailDuplicateBlock(<br>
+ MachineBasicBlock *BB, MachineBasicBlock *&LPred,<br>
+ MachineBasicBlock *LoopHeaderBB,<br>
+ BlockChain &Chain, BlockFilterSet *BlockFilter,<br>
+ MachineFunction::iterator &PrevUnplacedBlockIt);<br>
+ bool maybeTailDuplicateBlock(<wbr>MachineBasicBlock *BB, MachineBasicBlock *LPred,<br>
+ const BlockChain &Chain,<br>
+ BlockFilterSet *BlockFilter,<br>
+ MachineFunction::iterator &PrevUnplacedBlockIt,<br>
+ bool &DuplicatedToPred);<br>
bool<br>
hasBetterLayoutPredecessor(<wbr>MachineBasicBlock *BB, MachineBasicBlock *Succ,<br>
BlockChain &SuccChain, BranchProbability SuccProb,<br>
@@ -323,7 +375,7 @@ class MachineBlockPlacement : public Mac<br>
SmallPtrSetImpl<BlockChain *> &UpdatedPreds,<br>
const BlockFilterSet *BlockFilter);<br>
void buildChain(MachineBasicBlock *BB, BlockChain &Chain,<br>
- const BlockFilterSet *BlockFilter = nullptr);<br>
+ BlockFilterSet *BlockFilter = nullptr);<br>
MachineBasicBlock *findBestLoopTop(MachineLoop &L,<br>
const BlockFilterSet &LoopBlockSet);<br>
MachineBasicBlock *findBestLoopExit(MachineLoop &L,<br>
@@ -388,37 +440,49 @@ static std::string getBlockName(MachineB<br>
/// When a chain is being merged into the "placed" chain, this routine will<br>
/// quickly walk the successors of each block in the chain and mark them as<br>
/// having one fewer active predecessor. It also adds any successors of this<br>
-/// chain which reach the zero-predecessor state to the worklist passed in.<br>
+/// chain which reach the zero-predecessor state to the appropriate worklist.<br>
void MachineBlockPlacement::<wbr>markChainSuccessors(<br>
BlockChain &Chain, MachineBasicBlock *LoopHeaderBB,<br>
const BlockFilterSet *BlockFilter) {<br>
// Walk all the blocks in this chain, marking their successors as having<br>
// a predecessor placed.<br>
for (MachineBasicBlock *MBB : Chain) {<br>
- // Add any successors for which this is the only un-placed in-loop<br>
- // predecessor to the worklist as a viable candidate for CFG-neutral<br>
- // placement. No subsequent placement of this block will violate the CFG<br>
- // shape, so we get to use heuristics to choose a favorable placement.<br>
- for (MachineBasicBlock *Succ : MBB->successors()) {<br>
- if (BlockFilter && !BlockFilter->count(Succ))<br>
- continue;<br>
- BlockChain &SuccChain = *BlockToChain[Succ];<br>
- // Disregard edges within a fixed chain, or edges to the loop header.<br>
- if (&Chain == &SuccChain || Succ == LoopHeaderBB)<br>
- continue;<br>
+ markBlockSuccessors(Chain, MBB, LoopHeaderBB, BlockFilter);<br>
+ }<br>
+}<br>
<br>
- // This is a cross-chain edge that is within the loop, so decrement the<br>
- // loop predecessor count of the destination chain.<br>
- if (SuccChain.<wbr>UnscheduledPredecessors == 0 ||<br>
- --SuccChain.<wbr>UnscheduledPredecessors > 0)<br>
- continue;<br>
+/// \brief Mark a single block's successors as having one fewer preds.<br>
+///<br>
+/// Under normal circumstances, this is only called by markChainSuccessors,<br>
+/// but if a block that was to be placed is completely tail-duplicated away,<br>
+/// and was duplicated into the chain end, we need to redo markBlockSuccessors<br>
+/// for just that block.<br>
+void MachineBlockPlacement::<wbr>markBlockSuccessors(<br>
+ BlockChain &Chain, MachineBasicBlock *MBB, MachineBasicBlock *LoopHeaderBB,<br>
+ const BlockFilterSet *BlockFilter) {<br>
+ // Add any successors for which this is the only un-placed in-loop<br>
+ // predecessor to the worklist as a viable candidate for CFG-neutral<br>
+ // placement. No subsequent placement of this block will violate the CFG<br>
+ // shape, so we get to use heuristics to choose a favorable placement.<br>
+ for (MachineBasicBlock *Succ : MBB->successors()) {<br>
+ if (BlockFilter && !BlockFilter->count(Succ))<br>
+ continue;<br>
+ BlockChain &SuccChain = *BlockToChain[Succ];<br>
+ // Disregard edges within a fixed chain, or edges to the loop header.<br>
+ if (&Chain == &SuccChain || Succ == LoopHeaderBB)<br>
+ continue;<br>
<br>
- auto *MBB = *SuccChain.begin();<br>
- if (MBB->isEHPad())<br>
- EHPadWorkList.push_back(MBB);<br>
- else<br>
- BlockWorkList.push_back(MBB);<br>
- }<br>
+ // This is a cross-chain edge that is within the loop, so decrement the<br>
+ // loop predecessor count of the destination chain.<br>
+ if (SuccChain.<wbr>UnscheduledPredecessors == 0 ||<br>
+ --SuccChain.<wbr>UnscheduledPredecessors > 0)<br>
+ continue;<br>
+<br>
+ auto *NewBB = *SuccChain.begin();<br>
+ if (NewBB->isEHPad())<br>
+ EHPadWorkList.push_back(NewBB)<wbr>;<br>
+ else<br>
+ BlockWorkList.push_back(NewBB)<wbr>;<br>
}<br>
}<br>
<br>
@@ -902,7 +966,7 @@ void MachineBlockPlacement::<wbr>fillWorkList<br>
<br>
void MachineBlockPlacement::<wbr>buildChain(<br>
MachineBasicBlock *BB, BlockChain &Chain,<br>
- const BlockFilterSet *BlockFilter) {<br>
+ BlockFilterSet *BlockFilter) {<br>
assert(BB && "BB must not be null.\n");<br>
assert(BlockToChain[BB] == &Chain && "BlockToChainMap mis-match.\n");<br>
MachineFunction::iterator PrevUnplacedBlockIt = F->begin();<br>
@@ -937,6 +1001,17 @@ void MachineBlockPlacement::<wbr>buildChain(<br>
"layout successor until the CFG reduces\n");<br>
}<br>
<br>
+ // Placement may have changed tail duplication opportunities.<br>
+ // Check for that now.<br>
+ if (TailDupPlacement && BestSucc) {<br>
+ // If the chosen successor was duplicated into all its predecessors,<br>
+ // don't bother laying it out, just go round the loop again with BB as<br>
+ // the chain end.<br>
+ if (repeatedlyTailDuplicateBlock(<wbr>BestSucc, BB, LoopHeaderBB, Chain,<br>
+ BlockFilter, PrevUnplacedBlockIt))<br>
+ continue;<br>
+ }<br>
+<br>
// Place this block, updating the datastructures to reflect its placement.<br>
BlockChain &SuccChain = *BlockToChain[BestSucc];<br>
// Zero out UnscheduledPredecessors for the successor we're about to merge in case<br>
@@ -1718,6 +1793,175 @@ void MachineBlockPlacement::<wbr>alignBlocks(<br>
}<br>
}<br>
<br>
+/// Tail duplicate \p BB into (some) predecessors if profitable, repeating if<br>
+/// it was duplicated into its chain predecessor and removed.<br>
+/// \p BB - Basic block that may be duplicated.<br>
+///<br>
+/// \p LPred - Chosen layout predecessor of \p BB.<br>
+/// Updated to be the chain end if LPred is removed.<br>
+/// \p Chain - Chain to which \p LPred belongs, and \p BB will belong.<br>
+/// \p BlockFilter - Set of blocks that belong to the loop being laid out.<br>
+/// Used to identify which blocks to update predecessor<br>
+/// counts.<br>
+/// \p PrevUnplacedBlockIt - Iterator pointing to the last block that was<br>
+/// chosen in the given order due to unnatural CFG<br>
+/// only needed if \p BB is removed and<br>
+/// \p PrevUnplacedBlockIt pointed to \p BB.<br>
+/// @return true if \p BB was removed.<br>
+bool MachineBlockPlacement::<wbr>repeatedlyTailDuplicateBlock(<br>
+ MachineBasicBlock *BB, MachineBasicBlock *&LPred,<br>
+ MachineBasicBlock *LoopHeaderBB,<br>
+ BlockChain &Chain, BlockFilterSet *BlockFilter,<br>
+ MachineFunction::iterator &PrevUnplacedBlockIt) {<br>
+ bool Removed, DuplicatedToLPred;<br>
+ bool DuplicatedToOriginalLPred;<br>
+ Removed = maybeTailDuplicateBlock(BB, LPred, Chain, BlockFilter,<br>
+ PrevUnplacedBlockIt,<br>
+ DuplicatedToLPred);<br>
+ if (!Removed)<br>
+ return false;<br>
+ DuplicatedToOriginalLPred = DuplicatedToLPred;<br>
+ // Iteratively try to duplicate again. It can happen that a block that is<br>
+ // duplicated into is still small enough to be duplicated again.<br>
+ // No need to call markBlockSuccessors in this case, as the blocks being<br>
+ // duplicated from here on are already scheduled.<br>
+ // Note that DuplicatedToLPred always implies Removed.<br>
+ while (DuplicatedToLPred) {<br>
+ assert (Removed && "Block must have been removed to be duplicated into its "<br>
+ "layout predecessor.");<br>
+ MachineBasicBlock *DupBB, *DupPred;<br>
+ // The removal callback causes Chain.end() to be updated when a block is<br>
+ // removed. On the first pass through the loop, the chain end should be the<br>
+ // same as it was on function entry. On subsequent passes, because we are<br>
+ // duplicating the block at the end of the chain, if it is removed the<br>
+ // chain will have shrunk by one block.<br>
+ BlockChain::iterator ChainEnd = Chain.end();<br>
+ DupBB = *(--ChainEnd);<br>
+ // Now try to duplicate again.<br>
+ if (ChainEnd == Chain.begin())<br>
+ break;<br>
+ DupPred = *std::prev(ChainEnd);<br>
+ Removed = maybeTailDuplicateBlock(DupBB, DupPred, Chain, BlockFilter,<br>
+ PrevUnplacedBlockIt,<br>
+ DuplicatedToLPred);<br>
+ }<br>
+ // If BB was duplicated into LPred, it is now scheduled. But because it was<br>
+ // removed, markChainSuccessors won't be called for its chain. Instead we<br>
+ // call markBlockSuccessors for LPred to achieve the same effect. This must go<br>
+ // at the end because repeating the tail duplication can increase the number<br>
+ // of unscheduled predecessors.<br>
+ if (DuplicatedToOriginalLPred)<br>
+ markBlockSuccessors(Chain, LPred, LoopHeaderBB, BlockFilter);<br>
+<br>
+ LPred = *std::prev(Chain.end());<br>
+ return true;<br>
+}<br>
+<br>
+/// Tail duplicate \p BB into (some) predecessors if profitable.<br>
+/// \p BB - Basic block that may be duplicated<br>
+/// \p LPred - Chosen layout predecessor of \p BB<br>
+/// \p Chain - Chain to which \p LPred belongs, and \p BB will belong.<br>
+/// \p BlockFilter - Set of blocks that belong to the loop being laid out.<br>
+/// Used to identify which blocks to update predecessor<br>
+/// counts.<br>
+/// \p PrevUnplacedBlockIt - Iterator pointing to the last block that was<br>
+/// chosen in the given order due to unnatural CFG<br>
+/// only needed if \p BB is removed and<br>
+/// \p PrevUnplacedBlockIt pointed to \p BB.<br>
+/// \p DuplicatedToLPred - True if the block was duplicated into LPred. Will<br>
+/// only be true if the block was removed.<br>
+/// \return - True if the block was duplicated into all preds and removed.<br>
+bool MachineBlockPlacement::<wbr>maybeTailDuplicateBlock(<br>
+ MachineBasicBlock *BB, MachineBasicBlock *LPred,<br>
+ const BlockChain &Chain, BlockFilterSet *BlockFilter,<br>
+ MachineFunction::iterator &PrevUnplacedBlockIt,<br>
+ bool &DuplicatedToLPred) {<br>
+<br>
+ DuplicatedToLPred = false;<br>
+ DEBUG(dbgs() << "Redoing tail duplication for Succ#"<br>
+ << BB->getNumber() << "\n");<br>
+ bool IsSimple = TailDup.isSimpleBB(BB);<br>
+ // Blocks with single successors don't create additional fallthrough<br>
+ // opportunities. Don't duplicate them. TODO: When conditional exits are<br>
+ // analyzable, allow them to be duplicated.<br>
+ if (!IsSimple && BB->succ_size() == 1)<br>
+ return false;<br>
+ if (!TailDup.shouldTailDuplicate(<wbr>IsSimple, *BB))<br>
+ return false;<br>
+ // This has to be a callback because none of it can be done after<br>
+ // BB is deleted.<br>
+ bool Removed = false;<br>
+ auto RemovalCallback =<br>
+ [&](MachineBasicBlock *RemBB) {<br>
+ // Signal to outer function<br>
+ Removed = true;<br>
+<br>
+ // Conservative default.<br>
+ bool InWorkList = true;<br>
+ // Remove from the Chain and Chain Map<br>
+ if (BlockToChain.count(RemBB)) {<br>
+ BlockChain *Chain = BlockToChain[RemBB];<br>
+ InWorkList = Chain->UnscheduledPredecessors == 0;<br>
+ Chain->remove(RemBB);<br>
+ BlockToChain.erase(RemBB);<br>
+ }<br>
+<br>
+ // Handle the unplaced block iterator<br>
+ if (&(*PrevUnplacedBlockIt) == RemBB) {<br>
+ PrevUnplacedBlockIt++;<br>
+ }<br>
+<br>
+ // Handle the Work Lists<br>
+ if (InWorkList) {<br>
+ SmallVectorImpl<<wbr>MachineBasicBlock *> &RemoveList = BlockWorkList;<br>
+ if (RemBB->isEHPad())<br>
+ RemoveList = EHPadWorkList;<br>
+ RemoveList.erase(<br>
+ remove_if(RemoveList,<br>
+ [RemBB](MachineBasicBlock *BB) {return BB == RemBB;}),<br>
+ RemoveList.end());<br>
+ }<br>
+<br>
+ // Handle the filter set<br>
+ if (BlockFilter) {<br>
+ BlockFilter->erase(RemBB);<br>
+ }<br>
+<br>
+ // Remove the block from loop info.<br>
+ MLI->removeBlock(RemBB);<br>
+<br>
+ // TailDuplicator handles removing it from loops.<br>
+ DEBUG(dbgs() << "TailDuplicator deleted block: "<br>
+ << getBlockName(RemBB) << "\n");<br>
+ };<br>
+ auto RemovalCallbackRef =<br>
+ llvm::function_ref<void(<wbr>MachineBasicBlock*)>(<wbr>RemovalCallback);<br>
+<br>
+ SmallVector<MachineBasicBlock *, 8> DuplicatedPreds;<br>
+ TailDup.<wbr>tailDuplicateAndUpdate(<wbr>IsSimple, BB, LPred,<br>
+ &DuplicatedPreds, &RemovalCallbackRef);<br>
+<br>
+ // Update UnscheduledPredecessors to reflect tail-duplication.<br>
+ DuplicatedToLPred = false;<br>
+ for (MachineBasicBlock *Pred : DuplicatedPreds) {<br>
+ // We're only looking for unscheduled predecessors that match the filter.<br>
+ BlockChain* PredChain = BlockToChain[Pred];<br>
+ if (Pred == LPred)<br>
+ DuplicatedToLPred = true;<br>
+ if (Pred == LPred || (BlockFilter && !BlockFilter->count(Pred))<br>
+ || PredChain == &Chain)<br>
+ continue;<br>
+ for (MachineBasicBlock *NewSucc : Pred->successors()) {<br>
+ if (BlockFilter && !BlockFilter->count(NewSucc))<br>
+ continue;<br>
+ BlockChain *NewChain = BlockToChain[NewSucc];<br>
+ if (NewChain != &Chain && NewChain != PredChain)<br>
+ NewChain-><wbr>UnscheduledPredecessors++;<br>
+ }<br>
+ }<br>
+ return Removed;<br>
+}<br>
+<br>
bool MachineBlockPlacement::<wbr>runOnMachineFunction(<wbr>MachineFunction &MF) {<br>
if (skipFunction(*MF.getFunction(<wbr>)))<br>
return false;<br>
@@ -1734,6 +1978,13 @@ bool MachineBlockPlacement::<wbr>runOnMachine<br>
TII = MF.getSubtarget().<wbr>getInstrInfo();<br>
TLI = MF.getSubtarget().<wbr>getTargetLowering();<br>
MDT = &getAnalysis<<wbr>MachineDominatorTree>();<br>
+ if (TailDupPlacement) {<br>
+ unsigned TailDupSize = TailDuplicatePlacementThreshol<wbr>d;<br>
+ if (MF.getFunction()->optForSize(<wbr>))<br>
+ TailDupSize = 1;<br>
+ TailDup.initMF(MF, MBPI, /* LayoutMode */ true, TailDupSize);<br>
+ }<br>
+<br>
assert(BlockToChain.empty());<br>
<br>
buildCFGChains();<br>
@@ -1747,8 +1998,7 @@ bool MachineBlockPlacement::<wbr>runOnMachine<br>
BranchFoldPlacement;<br>
// No tail merging opportunities if the block number is less than four.<br>
if (MF.size() > 3 && EnableTailMerge) {<br>
- // Default to the standard tail-merge-size option.<br>
- unsigned TailMergeSize = 0;<br>
+ unsigned TailMergeSize = TailDuplicatePlacementThreshol<wbr>d + 1;<br>
BranchFolder BF(/*EnableTailMerge=*/true, /*CommonHoist=*/false, *MBFI,<br>
*MBPI, TailMergeSize);<br>
<br>
@@ -1757,6 +2007,8 @@ bool MachineBlockPlacement::<wbr>runOnMachine<br>
/*AfterBlockPlacement=*/true)) {<br>
// Redo the layout if tail merging creates/removes/moves blocks.<br>
BlockToChain.clear();<br>
+ // Must redo the dominator tree if blocks were changed.<br>
+ MDT->runOnMachineFunction(MF);<br>
ChainAllocator.DestroyAll();<br>
buildCFGChains();<br>
}<br>
<br>
Modified: llvm/trunk/lib/CodeGen/<wbr>TailDuplication.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/TailDuplication.cpp?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/<wbr>CodeGen/TailDuplication.cpp?<wbr>rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/lib/CodeGen/<wbr>TailDuplication.cpp (original)<br>
+++ llvm/trunk/lib/CodeGen/<wbr>TailDuplication.cpp Mon Oct 10 20:20:33 2016<br>
@@ -49,7 +49,7 @@ bool TailDuplicatePass::<wbr>runOnMachineFunc<br>
<br>
auto MBPI = &getAnalysis<<wbr>MachineBranchProbabilityInfo>(<wbr>);<br>
<br>
- Duplicator.initMF(MF, MBPI);<br>
+ Duplicator.initMF(MF, MBPI, /* LayoutMode */ false);<br>
<br>
bool MadeChange = false;<br>
while (Duplicator.<wbr>tailDuplicateBlocks())<br>
<br>
Modified: llvm/trunk/lib/CodeGen/<wbr>TailDuplicator.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/TailDuplicator.cpp?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/<wbr>CodeGen/TailDuplicator.cpp?<wbr>rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/lib/CodeGen/<wbr>TailDuplicator.cpp (original)<br>
+++ llvm/trunk/lib/CodeGen/<wbr>TailDuplicator.cpp Mon Oct 10 20:20:33 2016<br>
@@ -20,6 +20,7 @@<br>
#include "llvm/CodeGen/<wbr>MachineBranchProbabilityInfo.<wbr>h"<br>
#include "llvm/CodeGen/<wbr>MachineFunctionPass.h"<br>
#include "llvm/CodeGen/<wbr>MachineInstrBuilder.h"<br>
+#include "llvm/CodeGen/MachineLoopInfo.<wbr>h"<br>
#include "llvm/CodeGen/<wbr>MachineModuleInfo.h"<br>
#include "llvm/CodeGen/Passes.h"<br>
#include "llvm/IR/Function.h"<br>
@@ -64,7 +65,7 @@ static cl::opt<unsigned> TailDupLimit("t<br>
<br>
void TailDuplicator::initMF(<wbr>MachineFunction &MFin,<br>
const MachineBranchProbabilityInfo *MBPIin,<br>
- unsigned TailDupSizeIn) {<br>
+ bool LayoutModeIn, unsigned TailDupSizeIn) {<br>
MF = &MFin;<br>
TII = MF->getSubtarget().<wbr>getInstrInfo();<br>
TRI = MF->getSubtarget().<wbr>getRegisterInfo();<br>
@@ -75,6 +76,7 @@ void TailDuplicator::initMF(<wbr>MachineFunct<br>
<br>
assert(MBPI != nullptr && "Machine Branch Probability Info required");<br>
<br>
+ LayoutMode = LayoutModeIn;<br>
PreRegAlloc = MRI->isSSA();<br>
}<br>
<br>
@@ -127,18 +129,23 @@ static void VerifyPHIs(MachineFunction &<br>
/// Tail duplicate the block and cleanup.<br>
/// \p IsSimple - return value of isSimpleBB<br>
/// \p MBB - block to be duplicated<br>
+/// \p ForcedLayoutPred - If non-null, treat this block as the layout<br>
+/// predecessor, instead of using the ordering in MF<br>
/// \p DuplicatedPreds - if non-null, \p DuplicatedPreds will contain a list of<br>
/// all Preds that received a copy of \p MBB.<br>
+/// \p RemovalCallback - if non-null, called just before MBB is deleted.<br>
bool TailDuplicator::<wbr>tailDuplicateAndUpdate(<br>
bool IsSimple, MachineBasicBlock *MBB,<br>
- SmallVectorImpl<<wbr>MachineBasicBlock*> *DuplicatedPreds) {<br>
+ MachineBasicBlock *ForcedLayoutPred,<br>
+ SmallVectorImpl<<wbr>MachineBasicBlock*> *DuplicatedPreds,<br>
+ llvm::function_ref<void(<wbr>MachineBasicBlock *)> *RemovalCallback) {<br>
// Save the successors list.<br>
SmallSetVector<<wbr>MachineBasicBlock *, 8> Succs(MBB->succ_begin(),<br>
MBB->succ_end());<br>
<br>
SmallVector<MachineBasicBlock *, 8> TDBBs;<br>
SmallVector<MachineInstr *, 16> Copies;<br>
- if (!tailDuplicate(IsSimple, MBB, TDBBs, Copies))<br>
+ if (!tailDuplicate(IsSimple, MBB, ForcedLayoutPred, TDBBs, Copies))<br>
return false;<br>
<br>
++NumTails;<br>
@@ -156,7 +163,7 @@ bool TailDuplicator::<wbr>tailDuplicateAndUpd<br>
// If it is dead, remove it.<br>
if (isDead) {<br>
NumTailDupRemoved += MBB->size();<br>
- removeDeadBlock(MBB);<br>
+ removeDeadBlock(MBB, RemovalCallback);<br>
++NumDeadBlocks;<br>
}<br>
<br>
@@ -255,7 +262,7 @@ bool TailDuplicator::<wbr>tailDuplicateBlocks<br>
if (!shouldTailDuplicate(<wbr>IsSimple, *MBB))<br>
continue;<br>
<br>
- MadeChange |= tailDuplicateAndUpdate(<wbr>IsSimple, MBB);<br>
+ MadeChange |= tailDuplicateAndUpdate(<wbr>IsSimple, MBB, nullptr);<br>
}<br>
<br>
if (PreRegAlloc && TailDupVerify)<br>
@@ -514,8 +521,10 @@ void TailDuplicator::<wbr>updateSuccessorsPHI<br>
/// Determine if it is profitable to duplicate this block.<br>
bool TailDuplicator::<wbr>shouldTailDuplicate(bool IsSimple,<br>
MachineBasicBlock &TailBB) {<br>
- // Only duplicate blocks that end with unconditional branches.<br>
- if (TailBB.canFallThrough())<br>
+ // When doing tail-duplication during layout, the block ordering is in flux,<br>
+ // so canFallThrough returns a result based on incorrect information and<br>
+ // should just be ignored.<br>
+ if (!LayoutMode && TailBB.canFallThrough())<br>
return false;<br>
<br>
// Don't try to tail-duplicate single-block loops.<br>
@@ -735,7 +744,7 @@ bool TailDuplicator::<wbr>duplicateSimpleBB(<br>
<br>
bool TailDuplicator::<wbr>canTailDuplicate(<wbr>MachineBasicBlock *TailBB,<br>
MachineBasicBlock *PredBB) {<br>
- // EH edges are ignored by AnalyzeBranch.<br>
+ // EH edges are ignored by analyzeBranch.<br>
if (PredBB->succ_size() > 1)<br>
return false;<br>
<br>
@@ -750,7 +759,16 @@ bool TailDuplicator::<wbr>canTailDuplicate(Ma<br>
<br>
/// If it is profitable, duplicate TailBB's contents in each<br>
/// of its predecessors.<br>
+/// \p IsSimple result of isSimpleBB<br>
+/// \p TailBB Block to be duplicated.<br>
+/// \p ForcedLayoutPred When non-null, use this block as the layout predecessor<br>
+/// instead of the previous block in MF's order.<br>
+/// \p TDBBs A vector to keep track of all blocks tail-duplicated<br>
+/// into.<br>
+/// \p Copies A vector of copy instructions inserted. Used later to<br>
+/// walk all the inserted copies and remove redundant ones.<br>
bool TailDuplicator::tailDuplicate(<wbr>bool IsSimple, MachineBasicBlock *TailBB,<br>
+ MachineBasicBlock *ForcedLayoutPred,<br>
SmallVectorImpl<<wbr>MachineBasicBlock *> &TDBBs,<br>
SmallVectorImpl<MachineInstr *> &Copies) {<br>
DEBUG(dbgs() << "\n*** Tail-duplicating BB#" << TailBB->getNumber() << '\n');<br>
@@ -775,7 +793,12 @@ bool TailDuplicator::tailDuplicate(<wbr>bool<br>
continue;<br>
<br>
// Don't duplicate into a fall-through predecessor (at least for now).<br>
- if (PredBB->isLayoutSuccessor(<wbr>TailBB) && PredBB->canFallThrough())<br>
+ bool IsLayoutSuccessor = false;<br>
+ if (ForcedLayoutPred)<br>
+ IsLayoutSuccessor = (ForcedLayoutPred == PredBB);<br>
+ else if (PredBB->isLayoutSuccessor(<wbr>TailBB) && PredBB->canFallThrough())<br>
+ IsLayoutSuccessor = true;<br>
+ if (IsLayoutSuccessor)<br>
continue;<br>
<br>
DEBUG(dbgs() << "\nTail-duplicating into PredBB: " << *PredBB<br>
@@ -828,19 +851,27 @@ bool TailDuplicator::tailDuplicate(<wbr>bool<br>
// If TailBB was duplicated into all its predecessors except for the prior<br>
// block, which falls through unconditionally, move the contents of this<br>
// block into the prior block.<br>
- MachineBasicBlock *PrevBB = &*std::prev(TailBB-><wbr>getIterator());<br>
+ MachineBasicBlock *PrevBB = ForcedLayoutPred;<br>
+ if (!PrevBB)<br>
+ PrevBB = &*std::prev(TailBB-><wbr>getIterator());<br>
MachineBasicBlock *PriorTBB = nullptr, *PriorFBB = nullptr;<br>
SmallVector<MachineOperand, 4> PriorCond;<br>
// This has to check PrevBB->succ_size() because EH edges are ignored by<br>
- // AnalyzeBranch.<br>
+ // analyzeBranch.<br>
if (PrevBB->succ_size() == 1 &&<br>
// Layout preds are not always CFG preds. Check.<br>
*PrevBB->succ_begin() == TailBB &&<br>
!TII->analyzeBranch(*PrevBB, PriorTBB, PriorFBB, PriorCond, true) &&<br>
- PriorCond.empty() && !PriorTBB && TailBB->pred_size() == 1 &&<br>
+ PriorCond.empty() &&<br>
+ (!PriorTBB || PriorTBB == TailBB) &&<br>
+ TailBB->pred_size() == 1 &&<br>
!TailBB->hasAddressTaken()) {<br>
DEBUG(dbgs() << "\nMerging into block: " << *PrevBB<br>
<< "From MBB: " << *TailBB);<br>
+ // There may be a branch to the layout successor. This is unlikely but it<br>
+ // happens. The correct thing to do is to remove the branch before<br>
+ // duplicating the instructions in all cases.<br>
+ TII->removeBranch(*PrevBB);<br>
if (PreRegAlloc) {<br>
DenseMap<unsigned, RegSubRegPair> LocalVRMap;<br>
SmallVector<std::pair<<wbr>unsigned, RegSubRegPair>, 4> CopyInfos;<br>
@@ -864,6 +895,7 @@ bool TailDuplicator::tailDuplicate(<wbr>bool<br>
}<br>
appendCopies(PrevBB, CopyInfos, Copies);<br>
} else {<br>
+ TII->removeBranch(*PrevBB);<br>
// No PHIs to worry about, just splice the instructions over.<br>
PrevBB->splice(PrevBB->end(), TailBB, TailBB->begin(), TailBB->end());<br>
}<br>
@@ -936,10 +968,15 @@ void TailDuplicator::appendCopies(<wbr>Machin<br>
<br>
/// Remove the specified dead machine basic block from the function, updating<br>
/// the CFG.<br>
-void TailDuplicator::<wbr>removeDeadBlock(<wbr>MachineBasicBlock *MBB) {<br>
+void TailDuplicator::<wbr>removeDeadBlock(<br>
+ MachineBasicBlock *MBB,<br>
+ llvm::function_ref<void(<wbr>MachineBasicBlock *)> *RemovalCallback) {<br>
assert(MBB->pred_empty() && "MBB must be dead!");<br>
DEBUG(dbgs() << "\nRemoving MBB: " << *MBB);<br>
<br>
+ if (RemovalCallback)<br>
+ (*RemovalCallback)(MBB);<br>
+<br>
// Remove all successors.<br>
while (!MBB->succ_empty())<br>
MBB->removeSuccessor(MBB-><wbr>succ_end() - 1);<br>
<br>
Modified: llvm/trunk/test/CodeGen/<wbr>AArch64/arm64-extload-<wbr>knownzero.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AArch64/arm64-extload-knownzero.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/AArch64/arm64-extload-<wbr>knownzero.ll?rev=283842&r1=<wbr>283841&r2=283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/<wbr>AArch64/arm64-extload-<wbr>knownzero.ll (original)<br>
+++ llvm/trunk/test/CodeGen/<wbr>AArch64/arm64-extload-<wbr>knownzero.ll Mon Oct 10 20:20:33 2016<br>
@@ -12,7 +12,6 @@ bb1:<br>
%tmp2 = load i16, i16* %ptr, align 2<br>
br label %bb2<br>
bb2:<br>
-; CHECK: %bb2<br>
; CHECK-NOT: and {{w[0-9]+}}, [[REG]], #0xffff<br>
; CHECK: cmp [[REG]], #23<br>
%tmp3 = phi i16 [ 0, %entry ], [ %tmp2, %bb1 ]<br>
<br>
Modified: llvm/trunk/test/CodeGen/<wbr>AArch64/machine_cse.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AArch64/machine_cse.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/AArch64/machine_cse.<wbr>ll?rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/<wbr>AArch64/machine_cse.ll (original)<br>
+++ llvm/trunk/test/CodeGen/<wbr>AArch64/machine_cse.ll Mon Oct 10 20:20:33 2016<br>
@@ -1,4 +1,8 @@<br>
-; RUN: llc < %s -mtriple=aarch64-linux-gnuabi -O2 | FileCheck %s<br>
+; RUN: llc < %s -mtriple=aarch64-linux-gnuabi -O2 -tail-dup-placement=0 | FileCheck %s<br>
+; -tail-dup-placement causes tail duplication during layout. This breaks the<br>
+; assumptions of the test case as written (specifically, it creates an<br>
+; additional cmp instruction, creating a false positive), so we pass<br>
+; -tail-dup-placement=0 to restore the original behavior<br>
<br>
; marked as external to prevent possible optimizations<br>
@a = external global i32<br>
<br>
Added: llvm/trunk/test/CodeGen/<wbr>AArch64/tail-dup-repeat-<wbr>worklist.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AArch64/tail-dup-repeat-worklist.ll?rev=283842&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/AArch64/tail-dup-<wbr>repeat-worklist.ll?rev=283842&<wbr>view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/<wbr>AArch64/tail-dup-repeat-<wbr>worklist.ll (added)<br>
+++ llvm/trunk/test/CodeGen/<wbr>AArch64/tail-dup-repeat-<wbr>worklist.ll Mon Oct 10 20:20:33 2016<br>
@@ -0,0 +1,69 @@<br>
+; RUN: llc -O3 -o - -verify-machineinstrs %s | FileCheck %s<br>
+target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:<wbr>64-i128:128-n32:64-S128"<br>
+target triple = "aarch64-unknown-linux-gnu"<br>
+<br>
+%struct.s1 = type { %struct.s3*, %struct.s1* }<br>
+%struct.s2 = type opaque<br>
+%struct.s3 = type { i32 }<br>
+<br>
+; Function Attrs: nounwind<br>
+define internal fastcc i32 @repeated_dup_worklist(%<wbr>struct.s1** %pp1, %struct.s2* %p2, i32 %state, i1 %i1_1, i32 %i32_1) unnamed_addr #0 {<br>
+entry:<br>
+ br label %while.cond.outer<br>
+<br>
+; The loop gets laid out:<br>
+; %while.cond.outer<br>
+; %(null)<br>
+; %(null)<br>
+; %dup2<br>
+; and then %dup1 gets chosen as the next block.<br>
+; when dup2 is duplicated into dup1, %worklist could erroneously be placed on<br>
+; the worklist, because all of its current predecessors are now scheduled.<br>
+; However, after dup2 is tail-duplicated, %worklist can't be on the worklist<br>
+; because it now has unscheduled predecessors.q<br>
+; CHECK-LABEL: repeated_dup_worklist<br>
+; CHECK: // %entry<br>
+; CHECK: // %while.cond.outer<br>
+; first %(null) block<br>
+; CHECK: // in Loop:<br>
+; CHECK: ldr<br>
+; CHECK-NEXT: tbnz<br>
+; second %(null) block<br>
+; CHECK: // in Loop:<br>
+; CHECK: // %dup2<br>
+; CHECK: // %worklist<br>
+; CHECK: // %if.then96.i<br>
+while.cond.outer: ; preds = %dup1, %entry<br>
+ %<a href="http://progress.0.ph" rel="noreferrer" target="_blank">progress.0.ph</a> = phi i32 [ 0, %entry ], [ %progress.1, %dup1 ]<br>
+ %inc77 = add nsw i32 %<a href="http://progress.0.ph" rel="noreferrer" target="_blank">progress.0.ph</a>, 1<br>
+ %cmp = icmp slt i32 %<a href="http://progress.0.ph" rel="noreferrer" target="_blank">progress.0.ph</a>, %i32_1<br>
+ br i1 %cmp, label %dup2, label %dup1<br>
+<br>
+dup2: ; preds = %if.then96.i, %worklist, %while.cond.outer<br>
+ %<a href="http://progress.1.ph" rel="noreferrer" target="_blank">progress.1.ph</a> = phi i32 [ 0, %while.cond.outer ], [ %progress.1, %if.then96.i ], [ %progress.1, %worklist ]<br>
+ %.pr = load %struct.s1*, %struct.s1** %pp1, align 8<br>
+ br label %dup1<br>
+<br>
+dup1: ; preds = %dup2, %while.cond.outer<br>
+ %0 = phi %struct.s1* [ %.pr, %dup2 ], [ undef, %while.cond.outer ]<br>
+ %progress.1 = phi i32 [ %<a href="http://progress.1.ph" rel="noreferrer" target="_blank">progress.1.ph</a>, %dup2 ], [ %inc77, %while.cond.outer ]<br>
+ br i1 %i1_1, label %while.cond.outer, label %worklist<br>
+<br>
+worklist: ; preds = %dup1<br>
+ %snode94 = getelementptr inbounds %struct.s1, %struct.s1* %0, i64 0, i32 0<br>
+ %1 = load %struct.s3*, %struct.s3** %snode94, align 8<br>
+ %2 = getelementptr inbounds %struct.s3, %struct.s3* %1, i32 0, i32 0<br>
+ %3 = load i32, i32* %2, align 4<br>
+ %tobool95.i = icmp eq i32 %3, 0<br>
+ br i1 %tobool95.i, label %if.then96.i, label %dup2<br>
+<br>
+if.then96.i: ; preds = %worklist<br>
+ call fastcc void @free_s3(%struct.s2* %p2, %struct.s3* %1) #1<br>
+ br label %dup2<br>
+}<br>
+<br>
+; Function Attrs: nounwind<br>
+declare fastcc void @free_s3(%struct.s2*, %struct.s3*) unnamed_addr #0<br>
+<br>
+attributes #0 = { nounwind "correctly-rounded-divide-<wbr>sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="<wbr>false" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="<wbr>false" "no-trapping-math"="false" "stack-protector-buffer-size"=<wbr>"8" "target-cpu"="cortex-a57" "target-features"="+crc,+<wbr>crypto,+neon" "unsafe-fp-math"="false" "use-soft-float"="false" }<br>
+attributes #1 = { nounwind }<br>
<br>
Modified: llvm/trunk/test/CodeGen/ARM/<wbr>2011-03-23-PeepholeBug.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/ARM/2011-03-23-PeepholeBug.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/ARM/2011-03-23-<wbr>PeepholeBug.ll?rev=283842&r1=<wbr>283841&r2=283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/ARM/<wbr>2011-03-23-PeepholeBug.ll (original)<br>
+++ llvm/trunk/test/CodeGen/ARM/<wbr>2011-03-23-PeepholeBug.ll Mon Oct 10 20:20:33 2016<br>
@@ -25,7 +25,6 @@ bb1:<br>
br label %bb2<br>
<br>
bb2: ; preds = %bb1, %entry<br>
-; CHECK: bb2<br>
; CHECK: cmp [[REG]], #0<br>
; CHECK: ble<br>
%indvar = phi i32 [ %indvar.next, %bb1 ], [ 0, %entry ]<br>
<br>
Modified: llvm/trunk/test/CodeGen/<wbr>PowerPC/branch-opt.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/PowerPC/branch-opt.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/PowerPC/branch-opt.ll?<wbr>rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/<wbr>PowerPC/branch-opt.ll (original)<br>
+++ llvm/trunk/test/CodeGen/<wbr>PowerPC/branch-opt.ll Mon Oct 10 20:20:33 2016<br>
@@ -1,9 +1,21 @@<br>
-; RUN: llc -verify-machineinstrs < %s -march=ppc32 | \<br>
-; RUN: grep "b LBB.*" | count 4<br>
+; RUN: llc -verify-machineinstrs < %s -march=ppc32 | FileCheck %s<br>
<br>
target datalayout = "E-p:32:32"<br>
target triple = "powerpc-apple-darwin8.7.0"<br>
<br>
+;CHECK-LABEL: foo:<br>
+; There are 4 inner loops (%bb, %bb12, %bb25, %bb38) that all exit to %cond_next48<br>
+; The last (whichever it is) should have a fallthrough exit, and the other three<br>
+; need an unconditional branch. No other block should have an unconditional<br>
+; branch to cond_next48<br>
+; One of the blocks ends up with a loop exit block that gets a tail-duplicated copy<br>
+; of %cond_next48, so there should only be two unconditional branches.<br>
+<br>
+;CHECK: b LBB0_13<br>
+;CHECK: b LBB0_13<br>
+;CHECK-NOT: b LBB0_13<br>
+;CHECK: LBB0_13: ; %cond_next48<br>
+<br>
define void @foo(i32 %W, i32 %X, i32 %Y, i32 %Z) {<br>
entry:<br>
%tmp1 = and i32 %W, 1 ; <i32> [#uses=1]<br>
<br>
Modified: llvm/trunk/test/CodeGen/<wbr>PowerPC/sjlj.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/PowerPC/sjlj.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/PowerPC/sjlj.ll?rev=<wbr>283842&r1=283841&r2=283842&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/<wbr>PowerPC/sjlj.ll (original)<br>
+++ llvm/trunk/test/CodeGen/<wbr>PowerPC/sjlj.ll Mon Oct 10 20:20:33 2016<br>
@@ -74,24 +74,24 @@ return:<br>
; CHECK-DAG: std [[REGA]], [[OFF:[0-9]+]](31) # 8-byte Folded Spill<br>
; CHECK-DAG: std 1, 16([[REGA]])<br>
; CHECK-DAG: std 2, 24([[REGA]])<br>
-; CHECK: bcl 20, 31, .LBB1_5<br>
+; CHECK: bcl 20, 31, .LBB1_3<br>
; CHECK: li 3, 1<br>
-; CHECK: #EH_SjLj_Setup .LBB1_5<br>
+; CHECK: #EH_SjLj_Setup .LBB1_3<br>
; CHECK: b .LBB1_1<br>
<br>
-; CHECK: .LBB1_4:<br>
+; CHECK: .LBB1_3:<br>
+; CHECK: mflr [[REGL:[0-9]+]]<br>
+; CHECK: ld [[REG2:[0-9]+]], [[OFF]](31) # 8-byte Folded Reload<br>
+; CHECK: std [[REGL]], 8([[REG2]])<br>
+; CHECK: li 3, 0<br>
+<br>
+; CHECK: .LBB1_5:<br>
<br>
; CHECK: lfd<br>
; CHECK: lxvd2x<br>
; CHECK: ld<br>
; CHECK: blr<br>
<br>
-; CHECK: .LBB1_5:<br>
-; CHECK: mflr [[REGL:[0-9]+]]<br>
-; CHECK: ld [[REG2:[0-9]+]], [[OFF]](31) # 8-byte Folded Reload<br>
-; CHECK: std [[REGL]], 8([[REG2]])<br>
-; CHECK: li 3, 0<br>
-<br>
; CHECK-NOAV: @main<br>
; CHECK-NOAV-NOT: stxvd2x<br>
; CHECK-NOAV: bcl<br>
<br>
Added: llvm/trunk/test/CodeGen/<wbr>PowerPC/tail-dup-branch-to-<wbr>fallthrough.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/PowerPC/tail-dup-branch-to-fallthrough.ll?rev=283842&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/PowerPC/tail-dup-<wbr>branch-to-fallthrough.ll?rev=<wbr>283842&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/<wbr>PowerPC/tail-dup-branch-to-<wbr>fallthrough.ll (added)<br>
+++ llvm/trunk/test/CodeGen/<wbr>PowerPC/tail-dup-branch-to-<wbr>fallthrough.ll Mon Oct 10 20:20:33 2016<br>
@@ -0,0 +1,65 @@<br>
+; RUN: llc -O2 %s -o - | FileCheck %s<br>
+target datalayout = "E-m:e-i64:64-n32:64"<br>
+target triple = "powerpc64-unknown-linux-gnu"<br>
+<br>
+; Function Attrs: nounwind<br>
+declare void @llvm.lifetime.end(i64, i8* nocapture) #0<br>
+<br>
+declare void @f1()<br>
+declare void @f2()<br>
+declare void @f3()<br>
+declare void @f4()<br>
+<br>
+; Function Attrs: nounwind<br>
+; CHECK-LABEL: tail_dup_fallthrough_with_<wbr>branch<br>
+; CHECK: # %entry<br>
+; CHECK-NOT: # %{{[-_a-zA-Z0-9]+}}<br>
+; CHECK: # %entry<br>
+; CHECK-NOT: # %{{[-_a-zA-Z0-9]+}}<br>
+; CHECK: # %sw.0<br>
+; CHECK-NOT: # %{{[-_a-zA-Z0-9]+}}<br>
+; CHECK: # %sw.1<br>
+; CHECK-NOT: # %{{[-_a-zA-Z0-9]+}}<br>
+; CHECK: # %sw.default<br>
+; CHECK-NOT: # %{{[-_a-zA-Z0-9]+}}<br>
+; CHECK: # %if.then<br>
+; CHECK-NOT: # %{{[-_a-zA-Z0-9]+}}<br>
+; CHECK: # %if.else<br>
+; CHECK-NOT: # %{{[-_a-zA-Z0-9]+}}<br>
+; CHECK: .Lfunc_end0<br>
+define fastcc void @tail_dup_fallthrough_with_<wbr>branch(i32 %a, i1 %b) unnamed_addr #0 {<br>
+entry:<br>
+ switch i32 %a, label %sw.default [<br>
+ i32 0, label %sw.0<br>
+ i32 1, label %sw.1<br>
+ ]<br>
+<br>
+sw.0: ; preds = %entry<br>
+ call void @f1() #0<br>
+ br label %dup1<br>
+<br>
+sw.1: ; preds = %entry<br>
+ call void @f2() #0<br>
+ br label %dup1<br>
+<br>
+sw.default: ; preds = %entry<br>
+ br i1 %b, label %if.then, label %if.else<br>
+<br>
+if.then: ; preds = %sw.default<br>
+ call void @f3() #0<br>
+ br label %dup2<br>
+<br>
+if.else: ; preds = %sw.default<br>
+ call void @f4() #0<br>
+ br label %dup2<br>
+<br>
+dup1: ; preds = %sw.0, %sw.1<br>
+ call void @llvm.lifetime.end(i64 8, i8* nonnull undef) #0<br>
+ unreachable<br>
+<br>
+dup2: ; preds = %if.then, %if.else<br>
+ call void @llvm.lifetime.end(i64 8, i8* nonnull undef) #0<br>
+ unreachable<br>
+}<br>
+<br>
+attributes #0 = { nounwind }<br>
<br>
Added: llvm/trunk/test/CodeGen/<wbr>PowerPC/tail-dup-layout.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/PowerPC/tail-dup-layout.ll?rev=283842&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/PowerPC/tail-dup-<wbr>layout.ll?rev=283842&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/<wbr>PowerPC/tail-dup-layout.ll (added)<br>
+++ llvm/trunk/test/CodeGen/<wbr>PowerPC/tail-dup-layout.ll Mon Oct 10 20:20:33 2016<br>
@@ -0,0 +1,100 @@<br>
+; RUN: llc -outline-optional-branches -O2 < %s | FileCheck %s<br>
+target datalayout = "e-m:e-i64:64-n32:64"<br>
+target triple = "powerpc64le-grtev4-linux-gnu"<br>
+<br>
+; Intended layout:<br>
+; The outlining flag produces the layout<br>
+; test1<br>
+; test2<br>
+; test3<br>
+; test4<br>
+; exit<br>
+; optional1<br>
+; optional2<br>
+; optional3<br>
+; optional4<br>
+; Tail duplication puts test n+1 at the end of optional n<br>
+; so optional1 includes a copy of test2 at the end, and branches<br>
+; to test3 (at the top) or falls through to optional 2.<br>
+; The CHECK statements check for the whole string of tests and exit block,<br>
+; and then check that the correct test has been duplicated into the end of<br>
+; the optional blocks and that the optional blocks are in the correct order.<br>
+;CHECK-LABEL: f:<br>
+; test1 may have been merged with entry<br>
+;CHECK: mr [[TAGREG:[0-9]+]], 3<br>
+;CHECK: andi. {{[0-9]+}}, [[TAGREG]], 1<br>
+;CHECK-NEXT: bc 12, 1, [[OPT1LABEL:[._0-9A-Za-z]+]]<br>
+;CHECK-NEXT: [[TEST2LABEL:[._0-9A-Za-z]+]]: # %test2<br>
+;CHECK-NEXT: rlwinm. {{[0-9]+}}, [[TAGREG]], 0, 30, 30<br>
+;CHECK-NEXT: bne 0, [[OPT2LABEL:[._0-9A-Za-z]+]]<br>
+;CHECK-NEXT: [[TEST3LABEL:[._0-9A-Za-z]+]]: # %test3<br>
+;CHECK-NEXT: rlwinm. {{[0-9]+}}, [[TAGREG]], 0, 29, 29<br>
+;CHECK-NEXT: bne 0, .[[OPT3LABEL:[._0-9A-Za-z]+]]<br>
+;CHECK-NEXT: [[TEST4LABEL:[._0-9A-Za-z]+]]: # %test4<br>
+;CHECK-NEXT: rlwinm. {{[0-9]+}}, [[TAGREG]], 0, 28, 28<br>
+;CHECK-NEXT: bne 0, .[[OPT4LABEL:[._0-9A-Za-z]+]]<br>
+;CHECK-NEXT: [[EXITLABEL:[._0-9A-Za-z]+]]: # %exit<br>
+;CHECK: blr<br>
+;CHECK-NEXT: [[OPT1LABEL]]<br>
+;CHECK: rlwinm. {{[0-9]+}}, [[TAGREG]], 0, 30, 30<br>
+;CHECK-NEXT: beq 0, [[TEST3LABEL]]<br>
+;CHECK-NEXT: [[OPT2LABEL]]<br>
+;CHECK: rlwinm. {{[0-9]+}}, [[TAGREG]], 0, 29, 29<br>
+;CHECK-NEXT: beq 0, [[TEST4LABEL]]<br>
+;CHECK-NEXT: [[OPT3LABEL]]<br>
+;CHECK: rlwinm. {{[0-9]+}}, [[TAGREG]], 0, 28, 28<br>
+;CHECK-NEXT: beq 0, [[EXITLABEL]]<br>
+;CHECK-NEXT: [[OPT4LABEL]]<br>
+;CHECK: b [[EXITLABEL]]<br>
+<br>
+define void @f(i32 %tag) {<br>
+entry:<br>
+ br label %test1<br>
+test1:<br>
+ %tagbit1 = and i32 %tag, 1<br>
+ %tagbit1eq0 = icmp eq i32 %tagbit1, 0<br>
+ br i1 %tagbit1eq0, label %test2, label %optional1<br>
+optional1:<br>
+ call void @a()<br>
+ call void @a()<br>
+ call void @a()<br>
+ call void @a()<br>
+ br label %test2<br>
+test2:<br>
+ %tagbit2 = and i32 %tag, 2<br>
+ %tagbit2eq0 = icmp eq i32 %tagbit2, 0<br>
+ br i1 %tagbit2eq0, label %test3, label %optional2<br>
+optional2:<br>
+ call void @b()<br>
+ call void @b()<br>
+ call void @b()<br>
+ call void @b()<br>
+ br label %test3<br>
+test3:<br>
+ %tagbit3 = and i32 %tag, 4<br>
+ %tagbit3eq0 = icmp eq i32 %tagbit3, 0<br>
+ br i1 %tagbit3eq0, label %test4, label %optional3<br>
+optional3:<br>
+ call void @c()<br>
+ call void @c()<br>
+ call void @c()<br>
+ call void @c()<br>
+ br label %test4<br>
+test4:<br>
+ %tagbit4 = and i32 %tag, 8<br>
+ %tagbit4eq0 = icmp eq i32 %tagbit4, 0<br>
+ br i1 %tagbit4eq0, label %exit, label %optional4<br>
+optional4:<br>
+ call void @d()<br>
+ call void @d()<br>
+ call void @d()<br>
+ call void @d()<br>
+ br label %exit<br>
+exit:<br>
+ ret void<br>
+}<br>
+<br>
+declare void @a()<br>
+declare void @b()<br>
+declare void @c()<br>
+declare void @d()<br>
<br>
Modified: llvm/trunk/test/CodeGen/<wbr>WebAssembly/cfg-stackify.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/WebAssembly/cfg-stackify.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/WebAssembly/cfg-<wbr>stackify.ll?rev=283842&r1=<wbr>283841&r2=283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/<wbr>WebAssembly/cfg-stackify.ll (original)<br>
+++ llvm/trunk/test/CodeGen/<wbr>WebAssembly/cfg-stackify.ll Mon Oct 10 20:20:33 2016<br>
@@ -1,5 +1,5 @@<br>
; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-<wbr>return-opt -disable-block-placement -verify-machineinstrs -fast-isel=false | FileCheck %s<br>
-; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-<wbr>return-opt -verify-machineinstrs -fast-isel=false | FileCheck -check-prefix=OPT %s<br>
+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-<wbr>return-opt -tail-dup-placement=0 -verify-machineinstrs -fast-isel=false | FileCheck -check-prefix=OPT %s<br>
<br>
; Test the CFG stackifier pass.<br>
<br>
<br>
Modified: llvm/trunk/test/CodeGen/<wbr>WebAssembly/mem-intrinsics.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/WebAssembly/mem-intrinsics.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/WebAssembly/mem-<wbr>intrinsics.ll?rev=283842&r1=<wbr>283841&r2=283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/<wbr>WebAssembly/mem-intrinsics.ll (original)<br>
+++ llvm/trunk/test/CodeGen/<wbr>WebAssembly/mem-intrinsics.ll Mon Oct 10 20:20:33 2016<br>
@@ -1,4 +1,4 @@<br>
-; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-<wbr>return-opt | FileCheck %s<br>
+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-<wbr>return-opt -tail-dup-placement=0| FileCheck %s<br>
<br>
; Test memcpy, memmove, and memset intrinsics.<br>
<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/<wbr>block-placement.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/block-placement.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/block-placement.<wbr>ll?rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>block-placement.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>block-placement.ll Mon Oct 10 20:20:33 2016<br>
@@ -177,6 +177,12 @@ exit:<br>
ret i32 %sum<br>
}<br>
<br>
+; Tail duplication during layout can entirely remove body0 by duplicating it<br>
+; into the entry block and into body1. This is a good thing but it isn't what<br>
+; this test is looking for. So to make the blocks longer so they don't get<br>
+; duplicated, we add some calls to dummy.<br>
+declare void @dummy()<br>
+<br>
define i32 @test_loop_rotate(i32 %i, i32* %a) {<br>
; Check that we rotate conditional exits from the loop to the bottom of the<br>
; loop, eliminating unconditional branches to the top.<br>
@@ -194,6 +200,8 @@ body0:<br>
%base = phi i32 [ 0, %entry ], [ %sum, %body1 ]<br>
%next = add i32 %iv, 1<br>
%exitcond = icmp eq i32 %next, %i<br>
+ call void @dummy()<br>
+ call void @dummy()<br>
br i1 %exitcond, label %exit, label %body1<br>
<br>
body1:<br>
@@ -945,7 +953,7 @@ define void @benchmark_heapsort(i32 %n,<br>
; First rotated loop top.<br>
; CHECK: .p2align<br>
; CHECK: %while.end<br>
-; CHECK: %for.cond<br>
+; %for.cond gets completely tail-duplicated away.<br>
; CHECK: %if.then<br>
; CHECK: %if.else<br>
; CHECK: %if.end10<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/<wbr>cmov-into-branch.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/cmov-into-branch.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/cmov-into-branch.<wbr>ll?rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>cmov-into-branch.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>cmov-into-branch.ll Mon Oct 10 20:20:33 2016<br>
@@ -105,9 +105,11 @@ define i32 @weighted_select3(i32 %a, i32<br>
; CHECK-NEXT: testl %edi, %edi<br>
; CHECK-NEXT: je [[LABEL_BB6:.*]]<br>
; CHECK: movl %edi, %eax<br>
+; CHECK-NEXT: retq<br>
; CHECK: [[LABEL_BB6]]<br>
; CHECK-NEXT: movl %esi, %edi<br>
-; CHECK-NEXT: jmp<br>
+; CHECK-NEXT: movl %edi, %eax<br>
+; CHECK-NEXT: retq<br>
;<br>
%cmp = icmp ne i32 %a, 0<br>
%sel = select i1 %cmp, i32 %a, i32 %b, !prof !2<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/<wbr>fma-intrinsics-phi-213-to-231.<wbr>ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/fma-intrinsics-phi-213-to-231.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/fma-intrinsics-<wbr>phi-213-to-231.ll?rev=283842&<wbr>r1=283841&r2=283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>fma-intrinsics-phi-213-to-231.<wbr>ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>fma-intrinsics-phi-213-to-231.<wbr>ll Mon Oct 10 20:20:33 2016<br>
@@ -2,7 +2,7 @@<br>
<br>
; CHECK-LABEL: fmaddsubpd_loop_128:<br>
; CHECK: vfmaddsub231pd %xmm1, %xmm0, %xmm2<br>
-; CHECK: vmovaps %xmm2, %xmm0<br>
+; CHECK: vmovapd %xmm2, %xmm0<br>
; CHECK-NEXT: retq<br>
define <2 x double> @fmaddsubpd_loop_128(i32 %iter, <2 x double> %a, <2 x double> %b, <2 x double> %c) {<br>
entry:<br>
@@ -28,7 +28,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fmsubaddpd_loop_128:<br>
; CHECK: vfmsubadd231pd %xmm1, %xmm0, %xmm2<br>
-; CHECK: vmovaps %xmm2, %xmm0<br>
+; CHECK: vmovapd %xmm2, %xmm0<br>
; CHECK-NEXT: retq<br>
define <2 x double> @fmsubaddpd_loop_128(i32 %iter, <2 x double> %a, <2 x double> %b, <2 x double> %c) {<br>
entry:<br>
@@ -54,7 +54,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fmaddpd_loop_128:<br>
; CHECK: vfmadd231pd %xmm1, %xmm0, %xmm2<br>
-; CHECK: vmovaps %xmm2, %xmm0<br>
+; CHECK: vmovapd %xmm2, %xmm0<br>
; CHECK-NEXT: retq<br>
define <2 x double> @fmaddpd_loop_128(i32 %iter, <2 x double> %a, <2 x double> %b, <2 x double> %c) {<br>
entry:<br>
@@ -80,7 +80,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fmsubpd_loop_128:<br>
; CHECK: vfmsub231pd %xmm1, %xmm0, %xmm2<br>
-; CHECK: vmovaps %xmm2, %xmm0<br>
+; CHECK: vmovapd %xmm2, %xmm0<br>
; CHECK-NEXT: retq<br>
define <2 x double> @fmsubpd_loop_128(i32 %iter, <2 x double> %a, <2 x double> %b, <2 x double> %c) {<br>
entry:<br>
@@ -106,7 +106,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fnmaddpd_loop_128:<br>
; CHECK: vfnmadd231pd %xmm1, %xmm0, %xmm2<br>
-; CHECK: vmovaps %xmm2, %xmm0<br>
+; CHECK: vmovapd %xmm2, %xmm0<br>
; CHECK-NEXT: retq<br>
define <2 x double> @fnmaddpd_loop_128(i32 %iter, <2 x double> %a, <2 x double> %b, <2 x double> %c) {<br>
entry:<br>
@@ -132,7 +132,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fnmsubpd_loop_128:<br>
; CHECK: vfnmsub231pd %xmm1, %xmm0, %xmm2<br>
-; CHECK: vmovaps %xmm2, %xmm0<br>
+; CHECK: vmovapd %xmm2, %xmm0<br>
; CHECK-NEXT: retq<br>
define <2 x double> @fnmsubpd_loop_128(i32 %iter, <2 x double> %a, <2 x double> %b, <2 x double> %c) {<br>
entry:<br>
@@ -329,7 +329,7 @@ declare <4 x float> @llvm.x86.fma.vfnmsu<br>
<br>
; CHECK-LABEL: fmaddsubpd_loop_256:<br>
; CHECK: vfmaddsub231pd %ymm1, %ymm0, %ymm2<br>
-; CHECK: vmovaps %ymm2, %ymm0<br>
+; CHECK: vmovapd %ymm2, %ymm0<br>
; CHECK-NEXT: retq<br>
define <4 x double> @fmaddsubpd_loop_256(i32 %iter, <4 x double> %a, <4 x double> %b, <4 x double> %c) {<br>
entry:<br>
@@ -355,7 +355,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fmsubaddpd_loop_256:<br>
; CHECK: vfmsubadd231pd %ymm1, %ymm0, %ymm2<br>
-; CHECK: vmovaps %ymm2, %ymm0<br>
+; CHECK: vmovapd %ymm2, %ymm0<br>
; CHECK-NEXT: retq<br>
define <4 x double> @fmsubaddpd_loop_256(i32 %iter, <4 x double> %a, <4 x double> %b, <4 x double> %c) {<br>
entry:<br>
@@ -381,7 +381,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fmaddpd_loop_256:<br>
; CHECK: vfmadd231pd %ymm1, %ymm0, %ymm2<br>
-; CHECK: vmovaps %ymm2, %ymm0<br>
+; CHECK: vmovapd %ymm2, %ymm0<br>
; CHECK-NEXT: retq<br>
define <4 x double> @fmaddpd_loop_256(i32 %iter, <4 x double> %a, <4 x double> %b, <4 x double> %c) {<br>
entry:<br>
@@ -407,7 +407,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fmsubpd_loop_256:<br>
; CHECK: vfmsub231pd %ymm1, %ymm0, %ymm2<br>
-; CHECK: vmovaps %ymm2, %ymm0<br>
+; CHECK: vmovapd %ymm2, %ymm0<br>
; CHECK-NEXT: retq<br>
define <4 x double> @fmsubpd_loop_256(i32 %iter, <4 x double> %a, <4 x double> %b, <4 x double> %c) {<br>
entry:<br>
@@ -433,7 +433,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fnmaddpd_loop_256:<br>
; CHECK: vfnmadd231pd %ymm1, %ymm0, %ymm2<br>
-; CHECK: vmovaps %ymm2, %ymm0<br>
+; CHECK: vmovapd %ymm2, %ymm0<br>
; CHECK-NEXT: retq<br>
define <4 x double> @fnmaddpd_loop_256(i32 %iter, <4 x double> %a, <4 x double> %b, <4 x double> %c) {<br>
entry:<br>
@@ -459,7 +459,7 @@ for.end:<br>
<br>
; CHECK-LABEL: fnmsubpd_loop_256:<br>
; CHECK: vfnmsub231pd %ymm1, %ymm0, %ymm2<br>
-; CHECK: vmovaps %ymm2, %ymm0<br>
+; CHECK: vmovapd %ymm2, %ymm0<br>
; CHECK-NEXT: retq<br>
define <4 x double> @fnmsubpd_loop_256(i32 %iter, <4 x double> %a, <4 x double> %b, <4 x double> %c) {<br>
entry:<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/<wbr>fp-une-cmp.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/fp-une-cmp.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/fp-une-cmp.ll?rev=<wbr>283842&r1=283841&r2=283842&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>fp-une-cmp.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>fp-une-cmp.ll Mon Oct 10 20:20:33 2016<br>
@@ -56,11 +56,11 @@ define double @profile_metadata(double %<br>
; CHECK-NEXT: ucomisd %xmm1, %xmm0<br>
; CHECK-NEXT: jne .LBB1_1<br>
; CHECK-NEXT: jp .LBB1_1<br>
-; CHECK-NEXT: .LBB1_2: # %bb2<br>
+; CHECK-NEXT: # %bb2<br>
; CHECK-NEXT: retq<br>
; CHECK-NEXT: .LBB1_1: # %bb1<br>
; CHECK-NEXT: addsd {{.*}}(%rip), %xmm0<br>
-; CHECK-NEXT: jmp .LBB1_2<br>
+; CHECK-NEXT: retq<br>
<br>
entry:<br>
%mul = fmul double %x, %y<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/<wbr>pr11202.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/pr11202.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/pr11202.ll?rev=<wbr>283842&r1=283841&r2=283842&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>pr11202.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>pr11202.ll Mon Oct 10 20:20:33 2016<br>
@@ -15,5 +15,8 @@ l2:<br>
br label %l1<br>
}<br>
<br>
-; CHECK: .Ltmp0: # Address of block that was removed by CodeGen<br>
+; It is correct for either l1 or l2 to be removed.<br>
+; If l2 is removed, the message should be "Address of block that was removed by CodeGen"<br>
+; If l1 is removed, it should be "Block address taken."<br>
+; CHECK: .Ltmp0: # {{Address of block that was removed by CodeGen|Block address taken}}<br>
; CHECK: .quad .Ltmp0<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/<wbr>ragreedy-bug.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/ragreedy-bug.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/ragreedy-bug.ll?<wbr>rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>ragreedy-bug.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>ragreedy-bug.ll Mon Oct 10 20:20:33 2016<br>
@@ -3,16 +3,34 @@<br>
; This testing case is reduced from 197.parser prune_match function.<br>
; We make sure register copies are not generated on isupper.exit blocks.<br>
<br>
-; CHECK: isupper.exit<br>
+; isupper.exit and isupper.exit223 get tail-duplicated into all their<br>
+; predecessors.<br>
+; CHECK: cond.true.i.i<br>
; CHECK-NEXT: in Loop<br>
+; Mem-move<br>
+; CHECK-NEXT: movl<br>
+; CHECK-NEXT: andl<br>
; CHECK-NEXT: testl<br>
; CHECK-NEXT: jne<br>
-; CHECK: isupper.exit<br>
+; CHECK: cond.true.i.i217<br>
; CHECK-NEXT: in Loop<br>
+; Mem-move<br>
+; CHECK-NEXT: movl<br>
+; CHECK-NEXT: andl<br>
; CHECK-NEXT: testl<br>
; CHECK-NEXT: je<br>
+; CHECK: cond.false.i.i<br>
; CHECK: maskrune<br>
+; CHECK-NEXT: movzbl<br>
+; CHECK-NEXT: movzbl<br>
+; CHECK-NEXT: testl<br>
+; CHECK-NEXT: je<br>
+; CHECK: cond.false.i.i219<br>
; CHECK: maskrune<br>
+; CHECK-NEXT: movzbl<br>
+; CHECK-NEXT: movzbl<br>
+; CHECK-NEXT: testl<br>
+; CHECK-NEXT: jne<br>
<br>
%struct.List_o_links_struct = type { i32, i32, i32, %struct.List_o_links_struct* }<br>
%struct.Connector_struct = type { i16, i16, i8, i8, %struct.Connector_struct*, i8* }<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/<wbr>sse1.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/sse1.ll?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/sse1.ll?rev=<wbr>283842&r1=283841&r2=283842&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>sse1.ll (original)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>sse1.ll Mon Oct 10 20:20:33 2016<br>
@@ -58,21 +58,23 @@ define <4 x float> @vselect(<4 x float>*<br>
; X32-NEXT: je .LBB1_1<br>
; X32-NEXT: # BB#2: # %entry<br>
; X32-NEXT: xorps %xmm1, %xmm1<br>
-; X32-NEXT: jmp .LBB1_3<br>
+; X32-NEXT: cmpl $0, {{[0-9]+}}(%esp)<br>
+; X32-NEXT: jne .LBB1_5<br>
+; X32-NEXT: jmp .LBB1_4<br>
; X32-NEXT: .LBB1_1:<br>
; X32-NEXT: movss {{.*#+}} xmm1 = mem[0],zero,zero,zero<br>
-; X32-NEXT: .LBB1_3: # %entry<br>
; X32-NEXT: cmpl $0, {{[0-9]+}}(%esp)<br>
; X32-NEXT: je .LBB1_4<br>
-; X32-NEXT: # BB#5: # %entry<br>
+; X32-NEXT: .LBB1_5: # %entry<br>
; X32-NEXT: xorps %xmm2, %xmm2<br>
-; X32-NEXT: jmp .LBB1_6<br>
+; X32-NEXT: cmpl $0, {{[0-9]+}}(%esp)<br>
+; X32-NEXT: jne .LBB1_8<br>
+; X32-NEXT: jmp .LBB1_7<br>
; X32-NEXT: .LBB1_4:<br>
; X32-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero<br>
-; X32-NEXT: .LBB1_6: # %entry<br>
; X32-NEXT: cmpl $0, {{[0-9]+}}(%esp)<br>
; X32-NEXT: je .LBB1_7<br>
-; X32-NEXT: # BB#8: # %entry<br>
+; X32-NEXT: .LBB1_8: # %entry<br>
; X32-NEXT: xorps %xmm3, %xmm3<br>
; X32-NEXT: jmp .LBB1_9<br>
; X32-NEXT: .LBB1_7:<br>
@@ -95,21 +97,23 @@ define <4 x float> @vselect(<4 x float>*<br>
; X64-NEXT: je .LBB1_1<br>
; X64-NEXT: # BB#2: # %entry<br>
; X64-NEXT: xorps %xmm1, %xmm1<br>
-; X64-NEXT: jmp .LBB1_3<br>
+; X64-NEXT: testl %edx, %edx<br>
+; X64-NEXT: jne .LBB1_5<br>
+; X64-NEXT: jmp .LBB1_4<br>
; X64-NEXT: .LBB1_1:<br>
; X64-NEXT: movss {{.*#+}} xmm1 = mem[0],zero,zero,zero<br>
-; X64-NEXT: .LBB1_3: # %entry<br>
; X64-NEXT: testl %edx, %edx<br>
; X64-NEXT: je .LBB1_4<br>
-; X64-NEXT: # BB#5: # %entry<br>
+; X64-NEXT: .LBB1_5: # %entry<br>
; X64-NEXT: xorps %xmm2, %xmm2<br>
-; X64-NEXT: jmp .LBB1_6<br>
+; X64-NEXT: testl %r8d, %r8d<br>
+; X64-NEXT: jne .LBB1_8<br>
+; X64-NEXT: jmp .LBB1_7<br>
; X64-NEXT: .LBB1_4:<br>
; X64-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero<br>
-; X64-NEXT: .LBB1_6: # %entry<br>
; X64-NEXT: testl %r8d, %r8d<br>
; X64-NEXT: je .LBB1_7<br>
-; X64-NEXT: # BB#8: # %entry<br>
+; X64-NEXT: .LBB1_8: # %entry<br>
; X64-NEXT: xorps %xmm3, %xmm3<br>
; X64-NEXT: jmp .LBB1_9<br>
; X64-NEXT: .LBB1_7:<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/<wbr>tail-dup-merge-loop-headers.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/tail-dup-merge-loop-headers.ll?rev=283842&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/tail-dup-merge-<wbr>loop-headers.ll?rev=283842&<wbr>view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>tail-dup-merge-loop-headers.ll (added)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>tail-dup-merge-loop-headers.ll Mon Oct 10 20:20:33 2016<br>
@@ -0,0 +1,190 @@<br>
+; RUN: llc -O2 -o - %s | FileCheck %s<br>
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:<wbr>32:64-S128"<br>
+target triple = "x86_64-unknown-linux-gnu"<br>
+<br>
+; Function Attrs: nounwind uwtable<br>
+; CHECK-LABEL: tail_dup_merge_loops<br>
+; CHECK: # %entry<br>
+; CHECK-NOT: # %{{[a-zA-Z_]+}}<br>
+; CHECK: # %inner_loop_exit<br>
+; CHECK-NOT: # %{{[a-zA-Z_]+}}<br>
+; CHECK: # %inner_loop_latch<br>
+; CHECK-NOT: # %{{[a-zA-Z_]+}}<br>
+; CHECK: # %inner_loop_test<br>
+; CHECK-NOT: # %{{[a-zA-Z_]+}}<br>
+; CHECK: # %exit<br>
+define void @tail_dup_merge_loops(i32 %a, i8* %b, i8* %c) local_unnamed_addr #0 {<br>
+entry:<br>
+ %notlhs674.i = icmp eq i32 %a, 0<br>
+ br label %outer_loop_top<br>
+<br>
+outer_loop_top: ; preds = %inner_loop_exit, %entry<br>
+ %dst.0.ph.i = phi i8* [ %b, %entry ], [ %scevgep679.i, %inner_loop_exit ]<br>
+ br i1 %notlhs674.i, label %exit, label %inner_loop_preheader<br>
+<br>
+inner_loop_preheader: ; preds = %outer_loop_top<br>
+ br label %inner_loop_top<br>
+<br>
+inner_loop_top: ; preds = %inner_loop_latch, %inner_loop_preheader<br>
+ %dst.0.i = phi i8* [ %inc, %inner_loop_latch ], [ %dst.0.ph.i, %inner_loop_preheader ]<br>
+ %var = load i8, i8* %dst.0.i<br>
+ %tobool1.i = icmp slt i8 %var, 0<br>
+ br label %inner_loop_test<br>
+<br>
+inner_loop_test: ; preds = %inner_loop_top<br>
+ br i1 %tobool1.i, label %inner_loop_exit, label %inner_loop_latch<br>
+<br>
+inner_loop_exit: ; preds = %inner_loop_test<br>
+ %scevgep.i = getelementptr i8, i8* %dst.0.i, i64 1<br>
+ %scevgep679.i = getelementptr i8, i8* %scevgep.i, i64 0<br>
+ br label %outer_loop_top<br>
+<br>
+inner_loop_latch: ; preds = %inner_loop_test<br>
+ %cmp75.i = icmp ult i8* %dst.0.i, %c<br>
+ %inc = getelementptr i8, i8* %dst.0.i, i64 2<br>
+ br label %inner_loop_top<br>
+<br>
+exit: ; preds = %outer_loop_top<br>
+ ret void<br>
+}<br>
+<br>
+@.str.6 = external unnamed_addr constant [23 x i8], align 1<br>
+<br>
+; There is an erroneus check in LoopBase::addBasicBlockToLoop(<wbr>), where it<br>
+; assumes that the header block for a loop is unique.<br>
+; For most of compilation this assumption is true, but during layout we allow<br>
+; this assumption to be violated. The following code will trigger the bug:<br>
+<br>
+; The loops in question is eventually headed by the block shared_loop_header<br>
+;<br>
+; During layout The block labeled outer_loop_header gets tail-duplicated into<br>
+; outer_loop_latch, and into shared_preheader, and then removed. This leaves<br>
+; shared_loop_header as the header of both loops. The end result<br>
+; is that there are 2 valid loops, and that they share a header. If we re-ran<br>
+; the loop analysis, it would classify this as a single loop.<br>
+; So far this is fine as far as layout is concerned.<br>
+; After layout we tail merge blocks merge_other and merge_predecessor_split.<br>
+; We do this even though they share only a single instruction, because<br>
+; merge_predecessor_split falls through to their shared successor:<br>
+; outer_loop_latch.<br>
+; The rest of the blocks in the function are noise unfortunately. Bugpoint<br>
+; couldn't shrink the test any further.<br>
+<br>
+; CHECK-LABEL: loop_shared_header<br>
+; CHECK: # %entry<br>
+; CHECK: # %shared_preheader<br>
+; CHECK: # %shared_loop_header<br>
+; CHECK: # %inner_loop_body<br>
+; CHECK: # %merge_predecessor_split<br>
+; CHECK: # %outer_loop_latch<br>
+; CHECK: # %outer_loop_latch<br>
+; CHECK: # %cleanup<br>
+define i32 @loop_shared_header(i8* %exe, i32 %exesz, i32 %headsize, i32 %min, i32 %wwprva, i32 %e_lfanew, i8* readonly %wwp, i32 %wwpsz, i16 zeroext %sects) local_unnamed_addr #0 {<br>
+entry:<br>
+ %0 = load i32, i32* undef, align 4<br>
+ %mul = shl nsw i32 %0, 2<br>
+ br i1 undef, label %if.end19, label %cleanup<br>
+<br>
+if.end19: ; preds = %entry<br>
+ %conv = zext i32 %mul to i64<br>
+ %call = tail call i8* @cli_calloc(i64 %conv, i64 1)<br>
+ %1 = icmp eq i32 %exesz, 0<br>
+ %notrhs = icmp eq i32 %0, 0<br>
+ %or.cond117.not = or i1 %1, %notrhs<br>
+ %or.cond202 = or i1 %or.cond117.not, undef<br>
+ %cmp35 = icmp ult i8* undef, %exe<br>
+ %or.cond203 = or i1 %or.cond202, %cmp35<br>
+ br i1 %or.cond203, label %cleanup, label %if.end50<br>
+<br>
+if.end50: ; preds = %if.end19<br>
+ tail call void @llvm.memcpy.p0i8.p0i8.i64(i8* nonnull %call, i8* undef, i64 %conv, i32 1, i1 false)<br>
+ %cmp1.i.i = icmp ugt i32 %mul, 3<br>
+ br i1 %cmp1.i.i, label %shared_preheader, label %wunpsect.exit.thread.<wbr>loopexit391<br>
+<br>
+shared_preheader: ; preds = %if.end50<br>
+ br label %outer_loop_header<br>
+<br>
+outer_loop_header: ; preds = %outer_loop_latch, %shared_preheader<br>
+ %bits.1.i = phi i8 [ 32, %shared_preheader ], [ %bits.43.i, %outer_loop_latch ]<br>
+ %dst.0.ph.i = phi i8* [ undef, %shared_preheader ], [ %scevgep679.i, %outer_loop_latch ]<br>
+ %2 = icmp eq i32 undef, 0<br>
+ br i1 %2, label %while.cond.us1412.i, label %shared_loop_header<br>
+<br>
+while.cond.us1412.i: ; preds = %outer_loop_header<br>
+ %.pre.i = add i8 %bits.1.i, -1<br>
+ %tobool2.us1420.i = icmp eq i8 %.pre.i, 0<br>
+ %or.cond.us1421.i = or i1 undef, %tobool2.us1420.i<br>
+ br i1 %or.cond.us1421.i, label %if.end41.us1436.i, label %cleanup<br>
+<br>
+if.end41.us1436.i: ; preds = %while.cond.us1412.i<br>
+ unreachable<br>
+<br>
+shared_loop_header: ; preds = %dup_early2, %dup_early1<br>
+ %dst.0.i = phi i8* [ undef, %inner_loop_body ], [ %dst.0.ph.i, %outer_loop_header ], [ undef, %dead_block ]<br>
+ %cmp3.i1172.i = icmp ult i8* undef, %call<br>
+ br i1 %cmp3.i1172.i, label %wunpsect.exit.thread.<wbr>loopexit389, label %inner_loop_body<br>
+<br>
+inner_loop_body: ; preds = %shared_loop_header<br>
+ %3 = icmp slt i32 undef, 0<br>
+ br i1 %3, label %if.end96.i, label %shared_loop_header<br>
+<br>
+dead_block: ; preds = %inner_loop_body<br>
+ %cmp75.i = icmp ult i8* %dst.0.i, undef<br>
+ br label %shared_loop_header<br>
+<br>
+if.end96.i: ; preds = %inner_loop_body<br>
+ %cmp97.i = icmp ugt i32 undef, 2<br>
+ br i1 %cmp97.i, label %if.then99.i, label %if.end287.i<br>
+<br>
+if.then99.i: ; preds = %if.end96.i<br>
+ tail call void (i8*, ...) @cli_dbgmsg(i8* getelementptr inbounds ([23 x i8], [23 x i8]* @.str.6, i64 0, i64 0), i32 undef)<br>
+ br label %cleanup<br>
+<br>
+if.end287.i: ; preds = %if.end96.i<br>
+ %cmp291.i = icmp ne i32 undef, 1<br>
+ %conv294.i = select i1 %cmp291.i, i16 4, i16 3<br>
+ br i1 undef, label %if.end308.i, label %outer_loop_latch<br>
+<br>
+if.end308.i: ; preds = %if.end287.i<br>
+ br i1 undef, label %if.end335.i, label %merge_predecessor_split<br>
+<br>
+merge_predecessor_split: ; preds = %if.end308.i<br>
+ %4 = bitcast i8* undef to i32*<br>
+ br label %outer_loop_latch<br>
+<br>
+if.end335.i: ; preds = %if.end308.i<br>
+ br i1 undef, label %outer_loop_latch, label %merge_other<br>
+<br>
+merge_other: ; preds = %if.end335.i<br>
+ br label %outer_loop_latch<br>
+<br>
+outer_loop_latch: ; preds = %merge_other, %if.end335.i, %merge_predecessor_split, %if.end287.i<br>
+ %bits.43.i = phi i8 [ undef, %if.end287.i ], [ undef, %merge_other ], [ 32, %merge_predecessor_split ], [ 0, %if.end335.i ]<br>
+ %backsize.0.i = phi i16 [ %conv294.i, %if.end287.i ], [ 0, %merge_other ], [ 0, %merge_predecessor_split ], [ 0, %if.end335.i ]<br>
+ %5 = add i16 %backsize.0.i, -1<br>
+ %6 = zext i16 %5 to i64<br>
+ %scevgep.i = getelementptr i8, i8* %dst.0.ph.i, i64 1<br>
+ %scevgep679.i = getelementptr i8, i8* %scevgep.i, i64 %6<br>
+ br label %outer_loop_header<br>
+<br>
+wunpsect.exit.thread.<wbr>loopexit389: ; preds = %shared_loop_header<br>
+ unreachable<br>
+<br>
+wunpsect.exit.thread.<wbr>loopexit391: ; preds = %if.end50<br>
+ unreachable<br>
+<br>
+cleanup: ; preds = %if.then99.i, %while.cond.us1412.i, %if.end19, %entry<br>
+ %retval.0 = phi i32 [ 0, %if.then99.i ], [ 1, %entry ], [ 1, %if.end19 ], [ 1, %while.cond.us1412.i ]<br>
+ ret i32 %retval.0<br>
+}<br>
+<br>
+; Function Attrs: nounwind<br>
+declare void @cli_dbgmsg(i8*, ...) local_unnamed_addr #0<br>
+<br>
+; Function Attrs: nounwind<br>
+declare i8* @cli_calloc(i64, i64) local_unnamed_addr #0<br>
+<br>
+; Function Attrs: argmemonly nounwind<br>
+declare void @llvm.memcpy.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i32, i1) #1<br>
+attributes #0 = { nounwind }<br>
+attributes #1 = { argmemonly nounwind }<br>
<br>
Added: llvm/trunk/test/CodeGen/X86/<wbr>tail-dup-repeat.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/tail-dup-repeat.ll?rev=283842&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/tail-dup-repeat.<wbr>ll?rev=283842&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>tail-dup-repeat.ll (added)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>tail-dup-repeat.ll Mon Oct 10 20:20:33 2016<br>
@@ -0,0 +1,53 @@<br>
+; RUN: llc -O2 -tail-dup-placement-threshold=<wbr>4 -o - %s | FileCheck %s<br>
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:<wbr>32:64-S128"<br>
+target triple = "x86_64-unknown-linux-gnu"<br>
+<br>
+; Function Attrs: uwtable<br>
+; When tail-duplicating during placement, we work backward from blocks with<br>
+; multiple successors. In this case, the block dup1 gets duplicated into dup2<br>
+; and if.then64, and then the block dup2 gets duplicated into land.lhs.true<br>
+; and if.end70<br>
+; CHECK-LABEL: repeated_tail_dup:<br>
+define void @repeated_tail_dup(i1 %a1, i1 %a2, i32* %a4, i32* %a5, i8* %a6) #0 align 2 {<br>
+entry:<br>
+ br label %for.cond<br>
+<br>
+; CHECK: {{^}}.[[HEADER:LBB0_[1-9]]]: # %for.cond<br>
+for.cond: ; preds = %dup1, %entry<br>
+ br i1 %a1, label %land.lhs.true, label %if.end56<br>
+<br>
+land.lhs.true: ; preds = %for.cond<br>
+ store i32 10, i32* %a4, align 8<br>
+ br label %dup2<br>
+<br>
+if.end56: ; preds = %for.cond<br>
+ br i1 %a2, label %if.then64, label %if.end70<br>
+<br>
+if.then64: ; preds = %if.end56<br>
+ store i8 1, i8* %a6, align 1<br>
+ br label %dup1<br>
+<br>
+; CHECK: # %if.end70<br>
+; CHECK-NEXT: # in Loop:<br>
+; CHECK-NEXT: movl $12, (%rdx)<br>
+; CHECK-NEXT: movl $2, (%rcx)<br>
+; CHECK-NEXT: testl %eax, %eax<br>
+; CHECK-NEXT: je .[[HEADER]]<br>
+if.end70: ; preds = %if.end56<br>
+ store i32 12, i32* %a4, align 8<br>
+ br label %dup2<br>
+<br>
+dup2: ; preds = %if.end70, %land.lhs.true<br>
+ store i32 2, i32* %a5, align 4<br>
+ br label %dup1<br>
+<br>
+dup1: ; preds = %dup2, %if.then64<br>
+ %val = load i32, i32* %a4, align 8<br>
+ %switch = icmp ult i32 undef, 1<br>
+ br i1 %switch, label %for.cond, label %for.end<br>
+<br>
+for.end: ; preds = %dup1<br>
+ ret void<br>
+}<br>
+<br>
+attributes #0 = { uwtable }<br>
<br>
Modified: llvm/trunk/test/CodeGen/X86/<wbr>update-terminator.mir<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/update-terminator.mir?rev=283842&r1=283841&r2=283842&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/test/<wbr>CodeGen/X86/update-terminator.<wbr>mir?rev=283842&r1=283841&r2=<wbr>283842&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/CodeGen/X86/<wbr>update-terminator.mir (original)<br>
+++ llvm/trunk/test/CodeGen/X86/<wbr>update-terminator.mir Mon Oct 10 20:20:33 2016<br>
@@ -5,17 +5,30 @@<br>
@a = external global i16<br>
@b = external global i32<br>
<br>
+ declare void @dummy1()<br>
+ declare void @dummy2()<br>
+ declare void @dummy3()<br>
+<br>
; Function Attrs: nounwind<br>
define void @f2() {<br>
br i1 undef, label %bb1, label %bb3<br>
<br>
bb1:<br>
+ call void @dummy1()<br>
+ call void @dummy1()<br>
+ call void @dummy1()<br>
br i1 undef, label %bb2, label %bb2<br>
<br>
bb2:<br>
+ call void @dummy2()<br>
+ call void @dummy2()<br>
+ call void @dummy2()<br>
br label %bb4<br>
<br>
bb3:<br>
+ call void @dummy3()<br>
+ call void @dummy3()<br>
+ call void @dummy3()<br>
br label %bb2<br>
<br>
bb4:<br>
@@ -40,15 +53,24 @@ body: |<br>
bb.1:<br>
successors: %bb.2(100)<br>
<br>
+ CALL64pcrel32 @dummy1, csr_64, implicit %rsp, implicit-def %rsp<br>
+ CALL64pcrel32 @dummy1, csr_64, implicit %rsp, implicit-def %rsp<br>
+ CALL64pcrel32 @dummy1, csr_64, implicit %rsp, implicit-def %rsp<br>
JNE_1 %bb.2, implicit %eflags<br>
<br>
bb.2:<br>
successors: %bb.4(100)<br>
<br>
+ CALL64pcrel32 @dummy2, csr_64, implicit %rsp, implicit-def %rsp<br>
+ CALL64pcrel32 @dummy2, csr_64, implicit %rsp, implicit-def %rsp<br>
+ CALL64pcrel32 @dummy2, csr_64, implicit %rsp, implicit-def %rsp<br>
JMP_1 %bb.4<br>
<br>
bb.3:<br>
successors: %bb.2(100)<br>
+ CALL64pcrel32 @dummy3, csr_64, implicit %rsp, implicit-def %rsp<br>
+ CALL64pcrel32 @dummy3, csr_64, implicit %rsp, implicit-def %rsp<br>
+ CALL64pcrel32 @dummy3, csr_64, implicit %rsp, implicit-def %rsp<br>
JMP_1 %bb.2<br>
<br>
bb.4:<br>
<br>
<br>
______________________________<wbr>_________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br></div>