<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Jul 3, 2012, at 5:09 PM, Jakob Stoklund Olesen <<a href="mailto:stoklund@2pi.dk">stoklund@2pi.dk</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Author: stoklund<br>Date: Tue Jul 3 19:09:54 2012<br>New Revision: 159694<br><br>URL: <a href="http://llvm.org/viewvc/llvm-project?rev=159694&view=rev">http://llvm.org/viewvc/llvm-project?rev=159694&view=rev</a><br>Log:<br>Add an experimental early if-conversion pass, off by default.<br><br>This pass performs if-conversion on SSA form machine code by<br>speculatively executing both sides of the branch and using a cmov<br>instruction to select the result. This can help lower the number of<br>branch mispredictions on architectures like x86 that don't have<br>predicable instructions.<br><br>The current implementation is very aggressive, and causes regressions on<br>mosts tests. It needs good heuristics that have yet to be implemented.<br></blockquote><div><br></div><div>Nice new Pass!</div><div><br></div><div>You're running this after TailDup, probably because you want MachineDCE to run first. I'm not sure when EarlyTailDup is currently kicking in, but we generally want to try if-converting a pass before tailduping it.</div><div><br></div><div>Ideal pass order is: DCE, IfCvt, TailDup, ReDCE only where TailDup did something.</div><div><br></div><div>+for (MachineFunction::iterator MFI = MF.begin(), MFE = MF.end(); MFI != MFE;</div><div>+ ++MFI)</div><div>+ if (tryConvertIf(MFI))</div><div><br></div><div>How does this iterator work if you're deleting blocks? Maybe a comment.</div><div><br></div><div>I think it would be nice if CFG transforms don't depend on arbitrary block layout order.</div><div>Instead of using a worklist, would you consider a postorder domtree walk? That might solve two problems at once (stable order + nested if-conversion). The parent domtree node won't be invalidated by the if-conversion itself.</div><div><br></div><div>Speaking of which, Are you considering incremental DomTree update? It should be easy for if-conversion.</div><div><br></div><div>+ if (Tail->pred_size() != 2)</div><div>+ return false;</div><div><br></div><div>That doesn't look right. How can we handle simple nested cases like this?</div><div><br></div><div><font face="Menlo">A</font></div><div><font face="Menlo">|\</font></div><div><font face="Menlo">B \</font></div><div><font face="Menlo">|\ \</font></div><div><font face="Menlo">C E F</font></div><div><font face="Menlo">| | |</font></div><div><font face="Menlo"> \|/</font></div><div><font face="Menlo"> D</font></div><div><br></div><div>Taking it a step further, don't we want to handle simple "or" conditions?</div><div><br></div><div><font face="Menlo">A</font></div><div><font face="Menlo">| \</font></div><div><font face="Menlo">B F</font></div><div><font face="Menlo">|\ \</font></div><div><font face="Menlo">C E |</font></div><div><font face="Menlo">| \/</font></div><div><font face="Menlo">| G</font></div><div><font face="Menlo">| /</font></div><div><font face="Menlo">D</font></div><div><br></div><div>You're looking for simple patterns with single block on a path, so assuming prior passes have all aggressively merged fall-through blocks. Maybe this assumption should be commented.</div><div><br></div><div><div>+ bool SSAIfConv::canSpeculateInstrs(MachineBasicBlock *MBB) {</div><div>...</div></div><div>+ for (MachineBasicBlock::iterator I = MBB->begin(),</div><div>+ E = MBB->getFirstTerminator(); I != E; ++I) {</div><div><br></div><div>Dumb question: When is it safe to completely ignore MI terminators like this? Are we guaranteed that they don't write physical registers or have any other side effects? This would be a good place for a comment.</div><div><br></div><div>-Andy</div><div><br></div><blockquote type="cite"><br>Added:<br> llvm/trunk/lib/CodeGen/EarlyIfConversion.cpp<br>Modified:<br> llvm/trunk/include/llvm/CodeGen/Passes.h<br> llvm/trunk/include/llvm/InitializePasses.h<br> llvm/trunk/include/llvm/Target/TargetInstrInfo.h<br> llvm/trunk/lib/CodeGen/CMakeLists.txt<br> llvm/trunk/lib/CodeGen/CodeGen.cpp<br> llvm/trunk/lib/CodeGen/Passes.cpp<br><br>Modified: llvm/trunk/include/llvm/CodeGen/Passes.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/Passes.h?rev=159694&r1=159693&r2=159694&view=diff">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/Passes.h?rev=159694&r1=159693&r2=159694&view=diff</a><br>==============================================================================<br>--- llvm/trunk/include/llvm/CodeGen/Passes.h (original)<br>+++ llvm/trunk/include/llvm/CodeGen/Passes.h Tue Jul 3 19:09:54 2012<br>@@ -392,6 +392,10 @@<br> /// into tails of their predecessors.<br> extern char &TailDuplicateID;<br><br>+ /// EarlyIfConverter - This pass performs if-conversion on SSA form by<br>+ /// inserting cmov instructions.<br>+ extern char &EarlyIfConverterID;<br>+<br> /// IfConverter - This pass performs machine code if conversion.<br> extern char &IfConverterID;<br><br><br>Modified: llvm/trunk/include/llvm/InitializePasses.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/InitializePasses.h?rev=159694&r1=159693&r2=159694&view=diff">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/InitializePasses.h?rev=159694&r1=159693&r2=159694&view=diff</a><br>==============================================================================<br>--- llvm/trunk/include/llvm/InitializePasses.h (original)<br>+++ llvm/trunk/include/llvm/InitializePasses.h Tue Jul 3 19:09:54 2012<br>@@ -100,6 +100,7 @@<br> void initializeDomViewerPass(PassRegistry&);<br> void initializeDominanceFrontierPass(PassRegistry&);<br> void initializeDominatorTreePass(PassRegistry&);<br>+void initializeEarlyIfConverterPass(PassRegistry&);<br> void initializeEdgeBundlesPass(PassRegistry&);<br> void initializeEdgeProfilerPass(PassRegistry&);<br> void initializeExpandPostRAPass(PassRegistry&);<br><br>Modified: llvm/trunk/include/llvm/Target/TargetInstrInfo.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Target/TargetInstrInfo.h?rev=159694&r1=159693&r2=159694&view=diff">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Target/TargetInstrInfo.h?rev=159694&r1=159693&r2=159694&view=diff</a><br>==============================================================================<br>--- llvm/trunk/include/llvm/Target/TargetInstrInfo.h (original)<br>+++ llvm/trunk/include/llvm/Target/TargetInstrInfo.h Tue Jul 3 19:09:54 2012<br>@@ -369,6 +369,56 @@<br> return false;<br> }<br><br>+ /// canInsertSelect - Return true if it is possible to insert a select<br>+ /// instruction that chooses between TrueReg and FalseReg based on the<br>+ /// condition code in Cond.<br>+ ///<br>+ /// When successful, also return the latency in cycles from TrueReg,<br>+ /// FalseReg, and Cond to the destination register. The Cond latency should<br>+ /// compensate for a conditional branch being removed. For example, if a<br>+ /// conditional branch has a 3 cycle latency from the condition code read,<br>+ /// and a cmov instruction has a 2 cycle latency from the condition code<br>+ /// read, CondCycles should be returned as -1.<br>+ ///<br>+ /// @param MBB Block where select instruction would be inserted.<br>+ /// @param Cond Condition returned by AnalyzeBranch.<br>+ /// @param TrueReg Virtual register to select when Cond is true.<br>+ /// @param FalseReg Virtual register to select when Cond is false.<br>+ /// @param CondCycles Latency from Cond+Branch to select output.<br>+ /// @param TrueCycles Latency from TrueReg to select output.<br>+ /// @param FalseCycles Latency from FalseReg to select output.<br>+ virtual bool canInsertSelect(const MachineBasicBlock &MBB,<br>+ const SmallVectorImpl<MachineOperand> &Cond,<br>+ unsigned TrueReg, unsigned FalseReg,<br>+ int &CondCycles,<br>+ int &TrueCycles, int &FalseCycles) const {<br>+ return false;<br>+ }<br>+<br>+ /// insertSelect - Insert a select instruction into MBB before I that will<br>+ /// copy TrueReg to DstReg when Cond is true, and FalseReg to DstReg when<br>+ /// Cond is false.<br>+ ///<br>+ /// This function can only be called after canInsertSelect() returned true.<br>+ /// The condition in Cond comes from AnalyzeBranch, and it can be assumed<br>+ /// that the same flags or registers required by Cond are available at the<br>+ /// insertion point.<br>+ ///<br>+ /// @param MBB Block where select instruction should be inserted.<br>+ /// @param I Insertion point.<br>+ /// @param DL Source location for debugging.<br>+ /// @param DstReg Virtual register to be defined by select instruction.<br>+ /// @param Cond Condition as computed by AnalyzeBranch.<br>+ /// @param TrueReg Virtual register to copy when Cond is true.<br>+ /// @param FalseReg Virtual register to copy when Cons is false.<br>+ virtual void insertSelect(MachineBasicBlock &MBB,<br>+ MachineBasicBlock::iterator I, DebugLoc DL,<br>+ unsigned DstReg,<br>+ const SmallVectorImpl<MachineOperand> &Cond,<br>+ unsigned TrueReg, unsigned FalseReg) const {<br>+ llvm_unreachable("Target didn't implement TargetInstrInfo::insertSelect!");<br>+ }<br>+<br> /// copyPhysReg - Emit instructions to copy a pair of physical registers.<br> virtual void copyPhysReg(MachineBasicBlock &MBB,<br> MachineBasicBlock::iterator MI, DebugLoc DL,<br><br>Modified: llvm/trunk/lib/CodeGen/CMakeLists.txt<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CMakeLists.txt?rev=159694&r1=159693&r2=159694&view=diff">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CMakeLists.txt?rev=159694&r1=159693&r2=159694&view=diff</a><br>==============================================================================<br>--- llvm/trunk/lib/CodeGen/CMakeLists.txt (original)<br>+++ llvm/trunk/lib/CodeGen/CMakeLists.txt Tue Jul 3 19:09:54 2012<br>@@ -11,6 +11,7 @@<br> DeadMachineInstructionElim.cpp<br> DFAPacketizer.cpp<br> DwarfEHPrepare.cpp<br>+ EarlyIfConversion.cpp<br> EdgeBundles.cpp<br> ExecutionDepsFix.cpp<br> ExpandISelPseudos.cpp<br><br>Modified: llvm/trunk/lib/CodeGen/CodeGen.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CodeGen.cpp?rev=159694&r1=159693&r2=159694&view=diff">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CodeGen.cpp?rev=159694&r1=159693&r2=159694&view=diff</a><br>==============================================================================<br>--- llvm/trunk/lib/CodeGen/CodeGen.cpp (original)<br>+++ llvm/trunk/lib/CodeGen/CodeGen.cpp Tue Jul 3 19:09:54 2012<br>@@ -23,6 +23,7 @@<br> initializeCalculateSpillWeightsPass(Registry);<br> initializeCodePlacementOptPass(Registry);<br> initializeDeadMachineInstructionElimPass(Registry);<br>+ initializeEarlyIfConverterPass(Registry);<br> initializeExpandPostRAPass(Registry);<br> initializeExpandISelPseudosPass(Registry);<br> initializeFinalizeMachineBundlesPass(Registry);<br><br>Added: llvm/trunk/lib/CodeGen/EarlyIfConversion.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/EarlyIfConversion.cpp?rev=159694&view=auto">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/EarlyIfConversion.cpp?rev=159694&view=auto</a><br>==============================================================================<br>--- llvm/trunk/lib/CodeGen/EarlyIfConversion.cpp (added)<br>+++ llvm/trunk/lib/CodeGen/EarlyIfConversion.cpp Tue Jul 3 19:09:54 2012<br>@@ -0,0 +1,583 @@<br>+//===-- EarlyIfConversion.cpp - If-conversion on SSA form machine code ----===//<br>+//<br>+// The LLVM Compiler Infrastructure<br>+//<br>+// This file is distributed under the University of Illinois Open Source<br>+// License. See LICENSE.TXT for details.<br>+//<br>+//===----------------------------------------------------------------------===//<br>+//<br>+// Early if-conversion is for out-of-order CPUs that don't have a lot of<br>+// predicable instructions. The goal is to eliminate conditional branches that<br>+// may mispredict.<br>+//<br>+// Instructions from both sides of the branch are executed specutatively, and a<br>+// cmov instruction selects the result.<br>+//<br>+//===----------------------------------------------------------------------===//<br>+<br>+#define DEBUG_TYPE "early-ifcvt"<br>+#include "llvm/Function.h"<br>+#include "llvm/ADT/BitVector.h"<br>+#include "llvm/ADT/SetVector.h"<br>+#include "llvm/ADT/SmallPtrSet.h"<br>+#include "llvm/ADT/SparseSet.h"<br>+#include "llvm/CodeGen/MachineBranchProbabilityInfo.h"<br>+#include "llvm/CodeGen/MachineFunction.h"<br>+#include "llvm/CodeGen/MachineFunctionPass.h"<br>+#include "llvm/CodeGen/MachineRegisterInfo.h"<br>+#include "llvm/CodeGen/Passes.h"<br>+#include "llvm/Target/TargetInstrInfo.h"<br>+#include "llvm/Target/TargetRegisterInfo.h"<br>+#include "llvm/Support/CommandLine.h"<br>+#include "llvm/Support/Debug.h"<br>+#include "llvm/Support/raw_ostream.h"<br>+<br>+using namespace llvm;<br>+<br>+// Absolute maximum number of instructions allowed per speculated block.<br>+// This bypasses all other heuristics, so it should be set fairly high.<br>+static cl::opt<unsigned><br>+BlockInstrLimit("early-ifcvt-limit", cl::init(30), cl::Hidden,<br>+ cl::desc("Maximum number of instructions per speculated block."));<br>+<br>+// Stress testing mode - disable heuristics.<br>+static cl::opt<bool> Stress("stress-early-ifcvt", cl::Hidden,<br>+ cl::desc("Turn all knobs to 11"));<br>+<br>+typedef SmallSetVector<MachineBasicBlock*, 8> BlockSetVector;<br>+<br>+//===----------------------------------------------------------------------===//<br>+// SSAIfConv<br>+//===----------------------------------------------------------------------===//<br>+//<br>+// The SSAIfConv class performs if-conversion on SSA form machine code after<br>+// determining if it is possible. The class contains no heuristics, external<br>+// code should be used to determine when if-conversion is a good idea.<br>+//<br>+// SSAIfConv con convert both triangles and diamonds:<br>+//<br>+// Triangle: Head Diamond: Head<br>+// | \ / \<br>+// | \ / \<br>+// | [TF]BB FBB TBB<br>+// | / \ /<br>+// | / \ /<br>+// Tail Tail<br>+//<br>+// Instructions in the conditional blocks TBB and/or FBB are spliced into the<br>+// Head block, and phis in the Tail black are converted to select instruction.<br>+//<br>+namespace {<br>+class SSAIfConv {<br>+ const TargetInstrInfo *TII;<br>+ const TargetRegisterInfo *TRI;<br>+ MachineRegisterInfo *MRI;<br>+<br>+ /// The block containing the conditional branch.<br>+ MachineBasicBlock *Head;<br>+<br>+ /// The block containing phis after the if-then-else.<br>+ MachineBasicBlock *Tail;<br>+<br>+ /// The 'true' conditional block as determined by AnalyzeBranch.<br>+ MachineBasicBlock *TBB;<br>+<br>+ /// The 'false' conditional block as determined by AnalyzeBranch.<br>+ MachineBasicBlock *FBB;<br>+<br>+ /// isTriangle - When there is no 'else' block, either TBB or FBB will be<br>+ /// equal to Tail.<br>+ bool isTriangle() const { return TBB == Tail || FBB == Tail; }<br>+<br>+ /// The branch condition determined by AnalyzeBranch.<br>+ SmallVector<MachineOperand, 4> Cond;<br>+<br>+ /// Information about each phi in the Tail block.<br>+ struct PHIInfo {<br>+ MachineInstr *PHI;<br>+ unsigned TReg, FReg;<br>+ // Latencies from Cond+Branch, TReg, and FReg to DstReg.<br>+ int CondCycles, TCycles, FCycles;<br>+<br>+ PHIInfo(MachineInstr *phi)<br>+ : PHI(phi), TReg(0), FReg(0), CondCycles(0), TCycles(0), FCycles(0) {}<br>+ };<br>+<br>+ SmallVector<PHIInfo, 8> PHIs;<br>+<br>+ /// Instructions in Head that define values used by the conditional blocks.<br>+ /// The hoisted instructions must be inserted after these instructions.<br>+ SmallPtrSet<MachineInstr*, 8> InsertAfter;<br>+<br>+ /// Register units clobbered by the conditional blocks.<br>+ BitVector ClobberedRegUnits;<br>+<br>+ // Scratch pad for findInsertionPoint.<br>+ SparseSet<unsigned> LiveRegUnits;<br>+<br>+ /// Insertion point in Head for speculatively executed instructions form TBB<br>+ /// and FBB.<br>+ MachineBasicBlock::iterator InsertionPoint;<br>+<br>+ /// Return true if all non-terminator instructions in MBB can be safely<br>+ /// speculated.<br>+ bool canSpeculateInstrs(MachineBasicBlock *MBB);<br>+<br>+ /// Find a valid insertion point in Head.<br>+ bool findInsertionPoint();<br>+<br>+public:<br>+ /// runOnMachineFunction - Initialize per-function data structures.<br>+ void runOnMachineFunction(MachineFunction &MF) {<br>+ TII = MF.getTarget().getInstrInfo();<br>+ TRI = MF.getTarget().getRegisterInfo();<br>+ MRI = &MF.getRegInfo();<br>+ LiveRegUnits.clear();<br>+ LiveRegUnits.setUniverse(TRI->getNumRegUnits());<br>+ ClobberedRegUnits.clear();<br>+ ClobberedRegUnits.resize(TRI->getNumRegUnits());<br>+ }<br>+<br>+ /// canConvertIf - If the sub-CFG headed by MBB can be if-converted,<br>+ /// initialize the internal state, and return true.<br>+ bool canConvertIf(MachineBasicBlock *MBB);<br>+<br>+ /// convertIf - If-convert the last block passed to canConvertIf(), assuming<br>+ /// it is possible. Remove any erased blocks from WorkList<br>+ void convertIf(BlockSetVector &WorkList);<br>+};<br>+} // end anonymous namespace<br>+<br>+<br>+/// canSpeculateInstrs - Returns true if all the instructions in MBB can safely<br>+/// be speculated. The terminators are not considered.<br>+///<br>+/// If instructions use any values that are defined in the head basic block,<br>+/// the defining instructions are added to InsertAfter.<br>+///<br>+/// Any clobbered regunits are added to ClobberedRegUnits.<br>+///<br>+bool SSAIfConv::canSpeculateInstrs(MachineBasicBlock *MBB) {<br>+ // Reject any live-in physregs. It's probably CPSR/EFLAGS, and very hard to<br>+ // get right.<br>+ if (!MBB->livein_empty()) {<br>+ DEBUG(dbgs() << "BB#" << MBB->getNumber() << " has live-ins.\n");<br>+ return false;<br>+ }<br>+<br>+ unsigned InstrCount = 0;<br>+ for (MachineBasicBlock::iterator I = MBB->begin(),<br>+ E = MBB->getFirstTerminator(); I != E; ++I) {<br>+ if (I->isDebugValue())<br>+ continue;<br>+<br>+ if (++InstrCount > BlockInstrLimit && !Stress) {<br>+ DEBUG(dbgs() << "BB#" << MBB->getNumber() << " has more than "<br>+ << BlockInstrLimit << " instructions.\n");<br>+ return false;<br>+ }<br>+<br>+ // There shouldn't normally be any phis in a single-predecessor block.<br>+ if (I->isPHI()) {<br>+ DEBUG(dbgs() << "Can't hoist: " << *I);<br>+ return false;<br>+ }<br>+<br>+ // Don't speculate loads. Note that it may be possible and desirable to<br>+ // speculate GOT or constant pool loads that are guaranteed not to trap,<br>+ // but we don't support that for now.<br>+ if (I->mayLoad()) {<br>+ DEBUG(dbgs() << "Won't speculate load: " << *I);<br>+ return false;<br>+ }<br>+<br>+ // We never speculate stores, so an AA pointer isn't necessary.<br>+ bool DontMoveAcrossStore = true;<br>+ if (!I->isSafeToMove(TII, 0, DontMoveAcrossStore)) {<br>+ DEBUG(dbgs() << "Can't speculate: " << *I);<br>+ return false;<br>+ }<br>+<br>+ // Check for any dependencies on Head instructions.<br>+ for (MIOperands MO(I); MO.isValid(); ++MO) {<br>+ if (MO->isRegMask()) {<br>+ DEBUG(dbgs() << "Won't speculate regmask: " << *I);<br>+ return false;<br>+ }<br>+ if (!MO->isReg())<br>+ continue;<br>+ unsigned Reg = MO->getReg();<br>+<br>+ // Remember clobbered regunits.<br>+ if (MO->isDef() && TargetRegisterInfo::isPhysicalRegister(Reg))<br>+ for (MCRegUnitIterator Units(Reg, TRI); Units.isValid(); ++Units)<br>+ ClobberedRegUnits.set(*Units);<br>+<br>+ if (!MO->readsReg() || !TargetRegisterInfo::isVirtualRegister(Reg))<br>+ continue;<br>+ MachineInstr *DefMI = MRI->getVRegDef(Reg);<br>+ if (!DefMI || DefMI->getParent() != Head)<br>+ continue;<br>+ if (InsertAfter.insert(DefMI))<br>+ DEBUG(dbgs() << "BB#" << MBB->getNumber() << " depends on " << *DefMI);<br>+ if (DefMI->isTerminator()) {<br>+ DEBUG(dbgs() << "Can't insert instructions below terminator.\n");<br>+ return false;<br>+ }<br>+ }<br>+ }<br>+ return true;<br>+}<br>+<br>+<br>+/// Find an insertion point in Head for the speculated instructions. The<br>+/// insertion point must be:<br>+///<br>+/// 1. Before any terminators.<br>+/// 2. After any instructions in InsertAfter.<br>+/// 3. Not have any clobbered regunits live.<br>+///<br>+/// This function sets InsertionPoint and returns true when successful, it<br>+/// returns false if no valid insertion point could be found.<br>+///<br>+bool SSAIfConv::findInsertionPoint() {<br>+ // Keep track of live regunits before the current position.<br>+ // Only track RegUnits that are also in ClobberedRegUnits.<br>+ LiveRegUnits.clear();<br>+ SmallVector<unsigned, 8> Reads;<br>+ MachineBasicBlock::iterator FirstTerm = Head->getFirstTerminator();<br>+ MachineBasicBlock::iterator I = Head->end();<br>+ MachineBasicBlock::iterator B = Head->begin();<br>+ while (I != B) {<br>+ --I;<br>+ // Some of the conditional code depends in I.<br>+ if (InsertAfter.count(I)) {<br>+ DEBUG(dbgs() << "Can't insert code after " << *I);<br>+ return false;<br>+ }<br>+<br>+ // Update live regunits.<br>+ for (MIOperands MO(I); MO.isValid(); ++MO) {<br>+ // We're ignoring regmask operands. That is conservatively correct.<br>+ if (!MO->isReg())<br>+ continue;<br>+ unsigned Reg = MO->getReg();<br>+ if (!TargetRegisterInfo::isPhysicalRegister(Reg))<br>+ continue;<br>+ // I clobbers Reg, so it isn't live before I.<br>+ if (MO->isDef())<br>+ for (MCRegUnitIterator Units(Reg, TRI); Units.isValid(); ++Units)<br>+ LiveRegUnits.erase(*Units);<br>+ // Unless I reads Reg.<br>+ if (MO->readsReg())<br>+ Reads.push_back(Reg);<br>+ }<br>+ // Anything read by I is live before I.<br>+ while (!Reads.empty())<br>+ for (MCRegUnitIterator Units(Reads.pop_back_val(), TRI); Units.isValid();<br>+ ++Units)<br>+ if (ClobberedRegUnits.test(*Units))<br>+ LiveRegUnits.insert(*Units);<br>+<br>+ // We can't insert before a terminator.<br>+ if (I != FirstTerm && I->isTerminator())<br>+ continue;<br>+<br>+ // Some of the clobbered registers are live before I, not a valid insertion<br>+ // point.<br>+ if (!LiveRegUnits.empty()) {<br>+ DEBUG({<br>+ dbgs() << "Would clobber";<br>+ for (SparseSet<unsigned>::const_iterator<br>+ i = LiveRegUnits.begin(), e = LiveRegUnits.end(); i != e; ++i)<br>+ dbgs() << ' ' << PrintRegUnit(*i, TRI);<br>+ dbgs() << " live before " << *I;<br>+ });<br>+ continue;<br>+ }<br>+<br>+ // This is a valid insertion point.<br>+ InsertionPoint = I;<br>+ DEBUG(dbgs() << "Can insert before " << *I);<br>+ return true;<br>+ }<br>+ DEBUG(dbgs() << "No legal insertion point found.\n");<br>+ return false;<br>+}<br>+<br>+<br>+<br>+/// canConvertIf - analyze the sub-cfg rooted in MBB, and return true if it is<br>+/// a potential candidate for if-conversion. Fill out the internal state.<br>+///<br>+bool SSAIfConv::canConvertIf(MachineBasicBlock *MBB) {<br>+ Head = MBB;<br>+ TBB = FBB = Tail = 0;<br>+<br>+ if (Head->succ_size() != 2)<br>+ return false;<br>+ MachineBasicBlock *Succ0 = Head->succ_begin()[0];<br>+ MachineBasicBlock *Succ1 = Head->succ_begin()[1];<br>+<br>+ // Canonicalize so Succ0 has MBB as its single predecessor.<br>+ if (Succ0->pred_size() != 1)<br>+ std::swap(Succ0, Succ1);<br>+<br>+ if (Succ0->pred_size() != 1 || Succ0->succ_size() != 1)<br>+ return false;<br>+<br>+ // We could support additional Tail predecessors by updating phis instead of<br>+ // eliminating them. Let's see an example where it matters first.<br>+ Tail = Succ0->succ_begin()[0];<br>+ if (Tail->pred_size() != 2)<br>+ return false;<br>+<br>+ // This is not a triangle.<br>+ if (Tail != Succ1) {<br>+ // Check for a diamond. We won't deal with any critical edges.<br>+ if (Succ1->pred_size() != 1 || Succ1->succ_size() != 1 ||<br>+ Succ1->succ_begin()[0] != Tail)<br>+ return false;<br>+ DEBUG(dbgs() << "\nDiamond: BB#" << Head->getNumber()<br>+ << " -> BB#" << Succ0->getNumber()<br>+ << "/BB#" << Succ1->getNumber()<br>+ << " -> BB#" << Tail->getNumber() << '\n');<br>+<br>+ // Live-in physregs are tricky to get right when speculating code.<br>+ if (!Tail->livein_empty()) {<br>+ DEBUG(dbgs() << "Tail has live-ins.\n");<br>+ return false;<br>+ }<br>+ } else {<br>+ DEBUG(dbgs() << "\nTriangle: BB#" << Head->getNumber()<br>+ << " -> BB#" << Succ0->getNumber()<br>+ << " -> BB#" << Tail->getNumber() << '\n');<br>+ }<br>+<br>+ // This is a triangle or a diamond.<br>+ // If Tail doesn't have any phis, there must be side effects.<br>+ if (Tail->empty() || !Tail->front().isPHI()) {<br>+ DEBUG(dbgs() << "No phis in tail.\n");<br>+ return false;<br>+ }<br>+<br>+ // The branch we're looking to eliminate must be analyzable.<br>+ Cond.clear();<br>+ if (TII->AnalyzeBranch(*Head, TBB, FBB, Cond)) {<br>+ DEBUG(dbgs() << "Branch not analyzable.\n");<br>+ return false;<br>+ }<br>+<br>+ // This is weird, probably some sort of degenerate CFG.<br>+ if (!TBB) {<br>+ DEBUG(dbgs() << "AnalyzeBranch didn't find conditional branch.\n");<br>+ return false;<br>+ }<br>+<br>+ // AnalyzeBranch doesn't set FBB on a fall-through branch.<br>+ // Make sure it is always set.<br>+ FBB = TBB == Succ0 ? Succ1 : Succ0;<br>+<br>+ // Any phis in the tail block must be convertible to selects.<br>+ PHIs.clear();<br>+ MachineBasicBlock *TPred = TBB == Tail ? Head : TBB;<br>+ MachineBasicBlock *FPred = FBB == Tail ? Head : FBB;<br>+ for (MachineBasicBlock::iterator I = Tail->begin(), E = Tail->end();<br>+ I != E && I->isPHI(); ++I) {<br>+ PHIs.push_back(&*I);<br>+ PHIInfo &PI = PHIs.back();<br>+ // Find PHI operands corresponding to TPred and FPred.<br>+ for (unsigned i = 1; i != PI.PHI->getNumOperands(); i += 2) {<br>+ if (PI.PHI->getOperand(i+1).getMBB() == TPred)<br>+ PI.TReg = PI.PHI->getOperand(i).getReg();<br>+ if (PI.PHI->getOperand(i+1).getMBB() == FPred)<br>+ PI.FReg = PI.PHI->getOperand(i).getReg();<br>+ }<br>+ assert(TargetRegisterInfo::isVirtualRegister(PI.TReg) && "Bad PHI");<br>+ assert(TargetRegisterInfo::isVirtualRegister(PI.FReg) && "Bad PHI");<br>+<br>+ // Get target information.<br>+ if (!TII->canInsertSelect(*Head, Cond, PI.TReg, PI.FReg,<br>+ PI.CondCycles, PI.TCycles, PI.FCycles)) {<br>+ DEBUG(dbgs() << "Can't convert: " << *PI.PHI);<br>+ return false;<br>+ }<br>+ }<br>+<br>+ // Check that the conditional instructions can be speculated.<br>+ InsertAfter.clear();<br>+ ClobberedRegUnits.reset();<br>+ if (TBB != Tail && !canSpeculateInstrs(TBB))<br>+ return false;<br>+ if (FBB != Tail && !canSpeculateInstrs(FBB))<br>+ return false;<br>+<br>+ // Try to find a valid insertion point for the speculated instructions in the<br>+ // head basic block.<br>+ if (!findInsertionPoint())<br>+ return false;<br>+<br>+ return true;<br>+}<br>+<br>+<br>+static void eraseBlock(BlockSetVector &WorkList, MachineBasicBlock *MBB) {<br>+ WorkList.remove(MBB);<br>+ MBB->eraseFromParent();<br>+}<br>+<br>+<br>+/// convertIf - Execute the if conversion after canConvertIf has determined the<br>+/// feasibility.<br>+///<br>+/// Any basic blocks erased will also be removed from WorkList.<br>+///<br>+void SSAIfConv::convertIf(BlockSetVector &WorkList) {<br>+ assert(Head && Tail && TBB && FBB && "Call canConvertIf first.");<br>+<br>+ // Move all instructions into Head, except for the terminators.<br>+ if (TBB != Tail)<br>+ Head->splice(InsertionPoint, TBB, TBB->begin(), TBB->getFirstTerminator());<br>+ if (FBB != Tail)<br>+ Head->splice(InsertionPoint, FBB, FBB->begin(), FBB->getFirstTerminator());<br>+<br>+ MachineBasicBlock::iterator FirstTerm = Head->getFirstTerminator();<br>+ assert(FirstTerm != Head->end() && "No terminators");<br>+ DebugLoc HeadDL = FirstTerm->getDebugLoc();<br>+<br>+ // Convert all PHIs to select instructions inserted before FirstTerm.<br>+ for (unsigned i = 0, e = PHIs.size(); i != e; ++i) {<br>+ PHIInfo &PI = PHIs[i];<br>+ DEBUG(dbgs() << "If-converting " << *PI.PHI);<br>+ assert(PI.PHI->getNumOperands() == 5 && "Unexpected PHI operands.");<br>+ unsigned DstReg = PI.PHI->getOperand(0).getReg();<br>+ TII->insertSelect(*Head, FirstTerm, HeadDL, DstReg, Cond, PI.TReg, PI.FReg);<br>+ DEBUG(dbgs() << " --> " << *llvm::prior(FirstTerm));<br>+ PI.PHI->eraseFromParent();<br>+ PI.PHI = 0;<br>+ }<br>+<br>+ // Fix up the CFG, temporarily leave Head without any successors.<br>+ Head->removeSuccessor(TBB);<br>+ Head->removeSuccessor(FBB);<br>+ if (TBB != Tail)<br>+ TBB->removeSuccessor(Tail);<br>+ if (FBB != Tail)<br>+ FBB->removeSuccessor(Tail);<br>+<br>+ // Fix up Head's terminators.<br>+ // It should become a single branch or a fallthrough.<br>+ TII->RemoveBranch(*Head);<br>+<br>+ // Erase the now empty conditional blocks. It is likely that Head can fall<br>+ // through to Tail, and we can join the two blocks.<br>+ if (TBB != Tail)<br>+ eraseBlock(WorkList, TBB);<br>+ if (FBB != Tail)<br>+ eraseBlock(WorkList, FBB);<br>+<br>+ assert(Head->succ_empty() && "Additional head successors?");<br>+ if (Head->isLayoutSuccessor(Tail)) {<br>+ // Splice Tail onto the end of Head.<br>+ DEBUG(dbgs() << "Joining tail BB#" << Tail->getNumber()<br>+ << " into head BB#" << Head->getNumber() << '\n');<br>+ Head->splice(Head->end(), Tail,<br>+ Tail->begin(), Tail->end());<br>+ Head->transferSuccessorsAndUpdatePHIs(Tail);<br>+ eraseBlock(WorkList, Tail);<br>+<br>+ } else {<br>+ // We need a branch to Tail, let code placement work it out later.<br>+ DEBUG(dbgs() << "Converting to unconditional branch.\n");<br>+ SmallVector<MachineOperand, 0> EmptyCond;<br>+ TII->InsertBranch(*Head, Tail, 0, EmptyCond, HeadDL);<br>+ Head->addSuccessor(Tail);<br>+ }<br>+ DEBUG(dbgs() << *Head);<br>+}<br>+<br>+<br>+//===----------------------------------------------------------------------===//<br>+// EarlyIfConverter Pass<br>+//===----------------------------------------------------------------------===//<br>+<br>+namespace {<br>+class EarlyIfConverter : public MachineFunctionPass {<br>+ const TargetInstrInfo *TII;<br>+ const TargetRegisterInfo *TRI;<br>+ MachineRegisterInfo *MRI;<br>+ SSAIfConv IfConv;<br>+<br>+ // Worklist of head blocks to try for if-conversion.<br>+ BlockSetVector WorkList;<br>+<br>+public:<br>+ static char ID;<br>+ EarlyIfConverter() : MachineFunctionPass(ID) {}<br>+ void getAnalysisUsage(AnalysisUsage &AU) const;<br>+ bool runOnMachineFunction(MachineFunction &MF);<br>+<br>+private:<br>+ bool tryConvertIf(MachineBasicBlock*);<br>+};<br>+} // end anonymous namespace<br>+<br>+char EarlyIfConverter::ID = 0;<br>+char &llvm::EarlyIfConverterID = EarlyIfConverter::ID;<br>+<br>+INITIALIZE_PASS_BEGIN(EarlyIfConverter,<br>+ "early-ifcvt", "Early If Converter", false, false)<br>+INITIALIZE_PASS_DEPENDENCY(MachineBranchProbabilityInfo)<br>+INITIALIZE_PASS_END(EarlyIfConverter,<br>+ "early-ifcvt", "Early If Converter", false, false)<br>+<br>+void EarlyIfConverter::getAnalysisUsage(AnalysisUsage &AU) const {<br>+ AU.addRequired<MachineBranchProbabilityInfo>();<br>+ MachineFunctionPass::getAnalysisUsage(AU);<br>+}<br>+<br>+/// Attempt repeated if-conversion on MBB, return true if successful.<br>+/// Update WorkList with new opportunities.<br>+///<br>+bool EarlyIfConverter::tryConvertIf(MachineBasicBlock *MBB) {<br>+ if (!IfConv.canConvertIf(MBB))<br>+ return false;<br>+<br>+ // Repeatedly if-convert MBB, joining Head and Tail may expose more<br>+ // opportunities.<br>+ do IfConv.convertIf(WorkList);<br>+ while (IfConv.canConvertIf(MBB));<br>+<br>+ // It is possible that MBB is now itself a conditional block that can be<br>+ // if-converted.<br>+ if (MBB->pred_size() == 1 && MBB->succ_size() == 1)<br>+ WorkList.insert(MBB->pred_begin()[0]);<br>+ WorkList.remove(MBB);<br>+ return true;<br>+}<br>+<br>+<br>+bool EarlyIfConverter::runOnMachineFunction(MachineFunction &MF) {<br>+ DEBUG(dbgs() << "********** EARLY IF-CONVERSION **********\n"<br>+ << "********** Function: "<br>+ << ((Value*)MF.getFunction())->getName() << '\n');<br>+ TII = MF.getTarget().getInstrInfo();<br>+ TRI = MF.getTarget().getRegisterInfo();<br>+ MRI = &MF.getRegInfo();<br>+<br>+ bool Changed = false;<br>+ IfConv.runOnMachineFunction(MF);<br>+<br>+ for (MachineFunction::iterator MFI = MF.begin(), MFE = MF.end(); MFI != MFE;<br>+ ++MFI)<br>+ if (tryConvertIf(MFI))<br>+ Changed = true;<br>+<br>+ DEBUG(dbgs() << "Revisiting " << WorkList.size() << " blocks.\n");<br>+ while (!WorkList.empty())<br>+ tryConvertIf(WorkList.pop_back_val());<br>+<br>+ MF.verify(this, "After early if-conversion");<br>+ return Changed;<br>+}<br><br>Modified: llvm/trunk/lib/CodeGen/Passes.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/Passes.cpp?rev=159694&r1=159693&r2=159694&view=diff">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/Passes.cpp?rev=159694&r1=159693&r2=159694&view=diff</a><br>==============================================================================<br>--- llvm/trunk/lib/CodeGen/Passes.cpp (original)<br>+++ llvm/trunk/lib/CodeGen/Passes.cpp Tue Jul 3 19:09:54 2012<br>@@ -49,6 +49,8 @@<br> cl::desc("Disable Stack Slot Coloring"));<br> static cl::opt<bool> DisableMachineDCE("disable-machine-dce", cl::Hidden,<br> cl::desc("Disable Machine Dead Code Elimination"));<br>+static cl::opt<bool> EnableEarlyIfConversion("enable-early-ifcvt", cl::Hidden,<br>+ cl::desc("Enable Early If-conversion"));<br> static cl::opt<bool> DisableMachineLICM("disable-machine-licm", cl::Hidden,<br> cl::desc("Disable Machine LICM"));<br> static cl::opt<bool> DisableMachineCSE("disable-machine-cse", cl::Hidden,<br>@@ -154,6 +156,9 @@<br> if (StandardID == &DeadMachineInstructionElimID)<br> return applyDisable(TargetID, DisableMachineDCE);<br><br>+ if (StandardID == &EarlyIfConverterID)<br>+ return applyDisable(TargetID, !EnableEarlyIfConversion);<br>+<br> if (StandardID == &MachineLICMID)<br> return applyDisable(TargetID, DisableMachineLICM);<br><br>@@ -228,6 +233,9 @@<br> substitutePass(&EarlyTailDuplicateID, &TailDuplicateID);<br> substitutePass(&PostRAMachineLICMID, &MachineLICMID);<br><br>+ // Disable early if-conversion. Targets that are ready can enable it.<br>+ disablePass(&EarlyIfConverterID);<br>+<br> // Temporarily disable experimental passes.<br> substitutePass(&MachineSchedulerID, 0);<br> }<br>@@ -527,6 +535,7 @@<br> addPass(&DeadMachineInstructionElimID);<br> printAndVerify("After codegen DCE pass");<br><br>+ addPass(&EarlyIfConverterID);<br> addPass(&MachineLICMID);<br> addPass(&MachineCSEID);<br> addPass(&MachineSinkingID);<br><br><br>_______________________________________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br>http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits<br></blockquote></div><br></body></html>