<div dir="ltr">Is this being addressed?<div><br></div><div>This asserts on MSVC in a debug build because it uses debug iterators that check this kind of thing.</div><div dir="ltr"><div><br><br><div class="gmail_quote"><div dir="ltr">On Thu, May 25, 2017 at 9:05 AM Benjamin Kramer via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, May 25, 2017 at 2:51 PM, James Molloy via llvm-commits<br>
<<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br>
> Author: jamesm<br>
> Date: Thu May 25 07:51:11 2017<br>
> New Revision: 303850<br>
><br>
> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=303850&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=303850&view=rev</a><br>
> Log:<br>
> [GVNSink] GVNSink pass<br>
><br>
> This patch provides an initial prototype for a pass that sinks instructions based on GVN information, similar to GVNHoist. It is not yet ready for commiting but I've uploaded it to gather some initial thoughts.<br>
><br>
> This pass attempts to sink instructions into successors, reducing static<br>
> instruction count and enabling if-conversion.<br>
> We use a variant of global value numbering to decide what can be sunk.<br>
> Consider:<br>
><br>
> [ %a1 = add i32 %b, 1 ] [ %c1 = add i32 %d, 1 ]<br>
> [ %a2 = xor i32 %a1, 1 ] [ %c2 = xor i32 %c1, 1 ]<br>
> \ /<br>
> [ %e = phi i32 %a2, %c2 ]<br>
> [ add i32 %e, 4 ]<br>
><br>
> GVN would number %a1 and %c1 differently because they compute different<br>
> results - the VN of an instruction is a function of its opcode and the<br>
> transitive closure of its operands. This is the key property for hoisting<br>
> and CSE.<br>
><br>
> What we want when sinking however is for a numbering that is a function of<br>
> the *uses* of an instruction, which allows us to answer the question "if I<br>
> replace %a1 with %c1, will it contribute in an equivalent way to all<br>
> successive instructions?". The (new) PostValueTable class in GVN provides this<br>
> mapping.<br>
><br>
> This pass has some shown really impressive improvements especially for codesize already on internal benchmarks, so I have high hopes it can replace all the sinking logic in SimplifyCFG.<br>
><br>
> Differential revision: <a href="https://reviews.llvm.org/D24805" rel="noreferrer" target="_blank">https://reviews.llvm.org/D24805</a><br>
><br>
> Added:<br>
> llvm/trunk/lib/Transforms/Scalar/GVNSink.cpp<br>
> llvm/trunk/test/Transforms/GVNSink/<br>
> llvm/trunk/test/Transforms/GVNSink/dither.ll<br>
> llvm/trunk/test/Transforms/GVNSink/indirect-call.ll<br>
> llvm/trunk/test/Transforms/GVNSink/sink-common-code.ll<br>
> llvm/trunk/test/Transforms/GVNSink/struct.ll<br>
> Modified:<br>
> llvm/trunk/include/llvm/InitializePasses.h<br>
> llvm/trunk/include/llvm/Transforms/Scalar.h<br>
> llvm/trunk/include/llvm/Transforms/Scalar/GVN.h<br>
> llvm/trunk/include/llvm/Transforms/Utils/Local.h<br>
> llvm/trunk/lib/Transforms/IPO/PassManagerBuilder.cpp<br>
> llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt<br>
> llvm/trunk/lib/Transforms/Scalar/Scalar.cpp<br>
> llvm/trunk/lib/Transforms/Utils/Local.cpp<br>
> llvm/trunk/lib/Transforms/Utils/SimplifyCFG.cpp<br>
><br>
> Modified: llvm/trunk/include/llvm/InitializePasses.h<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/InitializePasses.h?rev=303850&r1=303849&r2=303850&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/InitializePasses.h?rev=303850&r1=303849&r2=303850&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/include/llvm/InitializePasses.h (original)<br>
> +++ llvm/trunk/include/llvm/InitializePasses.h Thu May 25 07:51:11 2017<br>
> @@ -144,6 +144,7 @@ void initializeGCMachineCodeAnalysisPass<br>
> void initializeGCModuleInfoPass(PassRegistry&);<br>
> void initializeGCOVProfilerLegacyPassPass(PassRegistry&);<br>
> void initializeGVNHoistLegacyPassPass(PassRegistry&);<br>
> +void initializeGVNSinkLegacyPassPass(PassRegistry&);<br>
> void initializeGVNLegacyPassPass(PassRegistry&);<br>
> void initializeGlobalDCELegacyPassPass(PassRegistry&);<br>
> void initializeGlobalMergePass(PassRegistry&);<br>
><br>
> Modified: llvm/trunk/include/llvm/Transforms/Scalar.h<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Scalar.h?rev=303850&r1=303849&r2=303850&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Scalar.h?rev=303850&r1=303849&r2=303850&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/include/llvm/Transforms/Scalar.h (original)<br>
> +++ llvm/trunk/include/llvm/Transforms/Scalar.h Thu May 25 07:51:11 2017<br>
> @@ -356,6 +356,13 @@ FunctionPass *createGVNHoistPass();<br>
><br>
> //===----------------------------------------------------------------------===//<br>
> //<br>
> +// GVNSink - This pass uses an "inverted" value numbering to decide the<br>
> +// similarity of expressions and sinks similar expressions into successors.<br>
> +//<br>
> +FunctionPass *createGVNSinkPass();<br>
> +<br>
> +//===----------------------------------------------------------------------===//<br>
> +//<br>
> // MergedLoadStoreMotion - This pass merges loads and stores in diamonds. Loads<br>
> // are hoisted into the header, while stores sink into the footer.<br>
> //<br>
><br>
> Modified: llvm/trunk/include/llvm/Transforms/Scalar/GVN.h<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Scalar/GVN.h?rev=303850&r1=303849&r2=303850&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Scalar/GVN.h?rev=303850&r1=303849&r2=303850&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/include/llvm/Transforms/Scalar/GVN.h (original)<br>
> +++ llvm/trunk/include/llvm/Transforms/Scalar/GVN.h Thu May 25 07:51:11 2017<br>
> @@ -238,7 +238,12 @@ struct GVNHoistPass : PassInfoMixin<GVNH<br>
> /// \brief Run the pass over the function.<br>
> PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM);<br>
> };<br>
> -<br>
> +/// \brief Uses an "inverted" value numbering to decide the similarity of<br>
> +/// expressions and sinks similar expressions into successors.<br>
> +struct GVNSinkPass : PassInfoMixin<GVNSinkPass> {<br>
> + /// \brief Run the pass over the function.<br>
> + PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM);<br>
> +};<br>
> }<br>
><br>
> #endif<br>
><br>
> Modified: llvm/trunk/include/llvm/Transforms/Utils/Local.h<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Utils/Local.h?rev=303850&r1=303849&r2=303850&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Utils/Local.h?rev=303850&r1=303849&r2=303850&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/include/llvm/Transforms/Utils/Local.h (original)<br>
> +++ llvm/trunk/include/llvm/Transforms/Utils/Local.h Thu May 25 07:51:11 2017<br>
> @@ -410,6 +410,14 @@ bool recognizeBSwapOrBitReverseIdiom(<br>
> void maybeMarkSanitizerLibraryCallNoBuiltin(CallInst *CI,<br>
> const TargetLibraryInfo *TLI);<br>
><br>
> +//===----------------------------------------------------------------------===//<br>
> +// Transform predicates<br>
> +//<br>
> +<br>
> +/// Given an instruction, is it legal to set operand OpIdx to a non-constant<br>
> +/// value?<br>
> +bool canReplaceOperandWithVariable(const Instruction *I, unsigned OpIdx);<br>
> +<br>
> } // End llvm namespace<br>
><br>
> #endif<br>
><br>
> Modified: llvm/trunk/lib/Transforms/IPO/PassManagerBuilder.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/IPO/PassManagerBuilder.cpp?rev=303850&r1=303849&r2=303850&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/IPO/PassManagerBuilder.cpp?rev=303850&r1=303849&r2=303850&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/Transforms/IPO/PassManagerBuilder.cpp (original)<br>
> +++ llvm/trunk/lib/Transforms/IPO/PassManagerBuilder.cpp Thu May 25 07:51:11 2017<br>
> @@ -155,6 +155,10 @@ static cl::opt<bool><br>
> cl::Hidden,<br>
> cl::desc("Enable the simple loop unswitch pass."));<br>
><br>
> +static cl::opt<bool> EnableGVNSink(<br>
> + "enable-gvn-sink", cl::init(false), cl::Hidden,<br>
> + cl::desc("Enable the GVN sinking pass (default = on)"));<br>
> +<br>
> PassManagerBuilder::PassManagerBuilder() {<br>
> OptLevel = 2;<br>
> SizeLevel = 0;<br>
> @@ -307,6 +311,11 @@ void PassManagerBuilder::addFunctionSimp<br>
> MPM.add(createEarlyCSEPass()); // Catch trivial redundancies<br>
> if (EnableGVNHoist)<br>
> MPM.add(createGVNHoistPass());<br>
> + if (EnableGVNSink) {<br>
> + MPM.add(createGVNSinkPass());<br>
> + MPM.add(createCFGSimplificationPass());<br>
> + }<br>
> +<br>
> // Speculative execution if the target has divergent branches; otherwise nop.<br>
> MPM.add(createSpeculativeExecutionIfHasBranchDivergencePass());<br>
> MPM.add(createJumpThreadingPass()); // Thread jumps.<br>
><br>
> Modified: llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt?rev=303850&r1=303849&r2=303850&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt?rev=303850&r1=303849&r2=303850&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt (original)<br>
> +++ llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt Thu May 25 07:51:11 2017<br>
> @@ -13,6 +13,7 @@ add_llvm_library(LLVMScalarOpts<br>
> GuardWidening.cpp<br>
> GVN.cpp<br>
> GVNHoist.cpp<br>
> + GVNSink.cpp<br>
> IVUsersPrinter.cpp<br>
> InductiveRangeCheckElimination.cpp<br>
> IndVarSimplify.cpp<br>
><br>
> Added: llvm/trunk/lib/Transforms/Scalar/GVNSink.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/GVNSink.cpp?rev=303850&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/GVNSink.cpp?rev=303850&view=auto</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/Transforms/Scalar/GVNSink.cpp (added)<br>
> +++ llvm/trunk/lib/Transforms/Scalar/GVNSink.cpp Thu May 25 07:51:11 2017<br>
> @@ -0,0 +1,870 @@<br>
> +//===- GVNSink.cpp - sink expressions into successors -------------------===//<br>
> +//<br>
> +// The LLVM Compiler Infrastructure<br>
> +//<br>
> +// This file is distributed under the University of Illinois Open Source<br>
> +// License. See LICENSE.TXT for details.<br>
> +//<br>
> +//===----------------------------------------------------------------------===//<br>
> +//<br>
> +/// \file GVNSink.cpp<br>
> +/// This pass attempts to sink instructions into successors, reducing static<br>
> +/// instruction count and enabling if-conversion.<br>
> +///<br>
> +/// We use a variant of global value numbering to decide what can be sunk.<br>
> +/// Consider:<br>
> +///<br>
> +/// [ %a1 = add i32 %b, 1 ] [ %c1 = add i32 %d, 1 ]<br>
> +/// [ %a2 = xor i32 %a1, 1 ] [ %c2 = xor i32 %c1, 1 ]<br>
> +/// \ /<br>
> +/// [ %e = phi i32 %a2, %c2 ]<br>
> +/// [ add i32 %e, 4 ]<br>
> +///<br>
> +///<br>
> +/// GVN would number %a1 and %c1 differently because they compute different<br>
> +/// results - the VN of an instruction is a function of its opcode and the<br>
> +/// transitive closure of its operands. This is the key property for hoisting<br>
> +/// and CSE.<br>
> +///<br>
> +/// What we want when sinking however is for a numbering that is a function of<br>
> +/// the *uses* of an instruction, which allows us to answer the question "if I<br>
> +/// replace %a1 with %c1, will it contribute in an equivalent way to all<br>
> +/// successive instructions?". The PostValueTable class in GVN provides this<br>
> +/// mapping.<br>
> +///<br>
> +//===----------------------------------------------------------------------===//<br>
> +<br>
> +#include "llvm/ADT/DenseMap.h"<br>
> +#include "llvm/ADT/DenseMapInfo.h"<br>
> +#include "llvm/ADT/DenseSet.h"<br>
> +#include "llvm/ADT/Hashing.h"<br>
> +#include "llvm/ADT/Optional.h"<br>
> +#include "llvm/ADT/PostOrderIterator.h"<br>
> +#include "llvm/ADT/SCCIterator.h"<br>
> +#include "llvm/ADT/SmallPtrSet.h"<br>
> +#include "llvm/ADT/Statistic.h"<br>
> +#include "llvm/ADT/StringExtras.h"<br>
> +#include "llvm/Analysis/GlobalsModRef.h"<br>
> +#include "llvm/Analysis/MemorySSA.h"<br>
> +#include "llvm/Analysis/PostDominators.h"<br>
> +#include "llvm/Analysis/TargetTransformInfo.h"<br>
> +#include "llvm/Analysis/ValueTracking.h"<br>
> +#include "llvm/IR/Instructions.h"<br>
> +#include "llvm/IR/Verifier.h"<br>
> +#include "llvm/Support/MathExtras.h"<br>
> +#include "llvm/Transforms/Scalar.h"<br>
> +#include "llvm/Transforms/Scalar/GVN.h"<br>
> +#include "llvm/Transforms/Scalar/GVNExpression.h"<br>
> +#include "llvm/Transforms/Utils/BasicBlockUtils.h"<br>
> +#include "llvm/Transforms/Utils/Local.h"<br>
> +#include <unordered_set><br>
> +using namespace llvm;<br>
> +<br>
> +#define DEBUG_TYPE "gvn-sink"<br>
> +<br>
> +STATISTIC(NumRemoved, "Number of instructions removed");<br>
> +<br>
> +namespace {<br>
> +<br>
> +static bool isMemoryInst(const Instruction *I) {<br>
> + return isa<LoadInst>(I) || isa<StoreInst>(I) ||<br>
> + (isa<InvokeInst>(I) && !cast<InvokeInst>(I)->doesNotAccessMemory()) ||<br>
> + (isa<CallInst>(I) && !cast<CallInst>(I)->doesNotAccessMemory());<br>
> +}<br>
> +<br>
> +/// Iterates through instructions in a set of blocks in reverse order from the<br>
> +/// first non-terminator. For example (assume all blocks have size n):<br>
> +/// LockstepReverseIterator I([B1, B2, B3]);<br>
> +/// *I-- = [B1[n], B2[n], B3[n]];<br>
> +/// *I-- = [B1[n-1], B2[n-1], B3[n-1]];<br>
> +/// *I-- = [B1[n-2], B2[n-2], B3[n-2]];<br>
> +/// ...<br>
> +///<br>
> +/// It continues until all blocks have been exhausted. Use \c getActiveBlocks()<br>
> +/// to<br>
> +/// determine which blocks are still going and the order they appear in the<br>
> +/// list returned by operator*.<br>
> +class LockstepReverseIterator {<br>
> + ArrayRef<BasicBlock *> Blocks;<br>
> + SmallPtrSet<BasicBlock *, 4> ActiveBlocks;<br>
> + SmallVector<Instruction *, 4> Insts;<br>
> + bool Fail;<br>
> +<br>
> +public:<br>
> + LockstepReverseIterator(ArrayRef<BasicBlock *> Blocks) : Blocks(Blocks) {<br>
> + reset();<br>
> + }<br>
> +<br>
> + void reset() {<br>
> + Fail = false;<br>
> + ActiveBlocks.clear();<br>
> + for (BasicBlock *BB : Blocks)<br>
> + ActiveBlocks.insert(BB);<br>
> + Insts.clear();<br>
> + for (BasicBlock *BB : Blocks) {<br>
> + if (BB->size() <= 1) {<br>
> + // Block wasn't big enough - only contained a terminator.<br>
> + ActiveBlocks.erase(BB);<br>
> + continue;<br>
> + }<br>
> + Insts.push_back(BB->getTerminator()->getPrevNode());<br>
> + }<br>
> + if (Insts.empty())<br>
> + Fail = true;<br>
> + }<br>
> +<br>
> + bool isValid() const { return !Fail; }<br>
> + ArrayRef<Instruction *> operator*() const { return Insts; }<br>
> + SmallPtrSet<BasicBlock *, 4> &getActiveBlocks() { return ActiveBlocks; }<br>
> +<br>
> + void restrictToBlocks(SmallPtrSetImpl<BasicBlock *> &Blocks) {<br>
> + for (auto II = Insts.begin(); II != Insts.end();) {<br>
> + if (std::find(Blocks.begin(), Blocks.end(), (*II)->getParent()) ==<br>
> + Blocks.end()) {<br>
> + ActiveBlocks.erase((*II)->getParent());<br>
> + II = Insts.erase(II);<br>
> + } else {<br>
> + ++II;<br>
> + }<br>
> + }<br>
> + }<br>
> +<br>
> + void operator--() {<br>
> + if (Fail)<br>
> + return;<br>
> + SmallVector<Instruction *, 4> NewInsts;<br>
> + for (auto *Inst : Insts) {<br>
> + if (Inst == &Inst->getParent()->front())<br>
> + ActiveBlocks.erase(Inst->getParent());<br>
> + else<br>
> + NewInsts.push_back(Inst->getPrevNode());<br>
> + }<br>
> + if (NewInsts.empty()) {<br>
> + Fail = true;<br>
> + return;<br>
> + }<br>
> + Insts = NewInsts;<br>
> + }<br>
> +};<br>
> +<br>
> +//===----------------------------------------------------------------------===//<br>
> +<br>
> +/// Candidate solution for sinking. There may be different ways to<br>
> +/// sink instructions, differing in the number of instructions sunk,<br>
> +/// the number of predecessors sunk from and the number of PHIs<br>
> +/// required.<br>
> +struct SinkingInstructionCandidate {<br>
> + unsigned NumBlocks;<br>
> + unsigned NumInstructions;<br>
> + unsigned NumPHIs;<br>
> + unsigned NumMemoryInsts;<br>
> + int Cost = -1;<br>
> + SmallVector<BasicBlock *, 4> Blocks;<br>
> +<br>
> + void calculateCost(unsigned NumOrigPHIs, unsigned NumOrigBlocks) {<br>
> + unsigned NumExtraPHIs = NumPHIs - NumOrigPHIs;<br>
> + unsigned SplitEdgeCost = (NumOrigBlocks > NumBlocks) ? 2 : 0;<br>
> + Cost = (NumInstructions * (NumBlocks - 1)) -<br>
> + (NumExtraPHIs *<br>
> + NumExtraPHIs) // PHIs are expensive, so make sure they're worth it.<br>
> + - SplitEdgeCost;<br>
> + }<br>
> + bool operator>=(const SinkingInstructionCandidate &Other) const {<br>
> + return Cost >= Other.Cost;<br>
> + }<br>
> +};<br>
> +<br>
> +llvm::raw_ostream &operator<<(llvm::raw_ostream &OS,<br>
> + const SinkingInstructionCandidate &C) {<br>
> + OS << "<Candidate Cost=" << C.Cost << " #Blocks=" << C.NumBlocks<br>
> + << " #Insts=" << C.NumInstructions << " #PHIs=" << C.NumPHIs << ">";<br>
> + return OS;<br>
> +}<br>
> +<br>
> +//===----------------------------------------------------------------------===//<br>
> +<br>
> +/// Describes a PHI node that may or may not exist. These track the PHIs<br>
> +/// that must be created if we sunk a sequence of instructions. It provides<br>
> +/// a hash function for efficient equality comparisons.<br>
> +class ModelledPHI {<br>
> + SmallVector<Value *, 4> Values;<br>
> + SmallVector<BasicBlock *, 4> Blocks;<br>
> +<br>
> +public:<br>
> + ModelledPHI() {}<br>
> + ModelledPHI(const PHINode *PN) {<br>
> + for (unsigned I = 0, E = PN->getNumIncomingValues(); I != E; ++I)<br>
> + Blocks.push_back(PN->getIncomingBlock(I));<br>
> + std::sort(Blocks.begin(), Blocks.end());<br>
> +<br>
> + // This assumes the PHI is already well-formed and there aren't conflicting<br>
> + // incoming values for the same block.<br>
> + for (auto *B : Blocks)<br>
> + Values.push_back(PN->getIncomingValueForBlock(B));<br>
> + }<br>
> + /// Create a dummy ModelledPHI that will compare unequal to any other ModelledPHI<br>
> + /// without the same ID.<br>
> + /// \note This is specifically for DenseMapInfo - do not use this!<br>
> + static ModelledPHI createDummy(unsigned ID) {<br>
> + ModelledPHI M;<br>
> + M.Values.push_back(reinterpret_cast<Value*>(ID));<br>
> + return M;<br>
> + }<br>
> +<br>
> + /// Create a PHI from an array of incoming values and incoming blocks.<br>
> + template <typename VArray, typename BArray><br>
> + ModelledPHI(const VArray &V, const BArray &B) {<br>
> + std::copy(V.begin(), V.end(), std::back_inserter(Values));<br>
> + std::copy(B.begin(), B.end(), std::back_inserter(Blocks));<br>
> + }<br>
> +<br>
> + /// Create a PHI from [I[OpNum] for I in Insts].<br>
> + template <typename BArray><br>
> + ModelledPHI(ArrayRef<Instruction *> Insts, unsigned OpNum, const BArray &B) {<br>
> + std::copy(B.begin(), B.end(), std::back_inserter(Blocks));<br>
> + for (auto *I : Insts)<br>
> + Values.push_back(I->getOperand(OpNum));<br>
> + }<br>
> +<br>
> + /// Restrict the PHI's contents down to only \c NewBlocks.<br>
> + /// \c NewBlocks must be a subset of \c this->Blocks.<br>
> + void restrictToBlocks(const SmallPtrSetImpl<BasicBlock *> &NewBlocks) {<br>
> + auto BI = Blocks.begin();<br>
> + auto VI = Values.begin();<br>
> + while (BI != Blocks.end()) {<br>
> + assert(VI != Values.end());<br>
> + if (std::find(NewBlocks.begin(), NewBlocks.end(), *BI) ==<br>
> + NewBlocks.end()) {<br>
> + BI = Blocks.erase(BI);<br>
> + VI = Values.erase(VI);<br>
> + } else {<br>
> + ++BI;<br>
> + ++VI;<br>
> + }<br>
> + }<br>
> + assert(Blocks.size() == NewBlocks.size());<br>
> + }<br>
> +<br>
> + ArrayRef<Value *> getValues() const { return Values; }<br>
> +<br>
> + bool areAllIncomingValuesSame() const {<br>
> + return all_of(Values, [&](Value *V) { return V == Values[0]; });<br>
> + }<br>
> + bool areAllIncomingValuesSameType() const {<br>
> + return all_of(<br>
> + Values, [&](Value *V) { return V->getType() == Values[0]->getType(); });<br>
> + }<br>
> + bool areAnyIncomingValuesConstant() const {<br>
> + return any_of(Values, [&](Value *V) { return isa<Constant>(V); });<br>
> + }<br>
> + // Hash functor<br>
> + unsigned hash() const {<br>
> + return (unsigned)hash_combine_range(Values.begin(), Values.end());<br>
> + }<br>
> + bool operator==(const ModelledPHI &Other) const {<br>
> + return Values == Other.Values && Blocks == Other.Blocks;<br>
> + }<br>
> +};<br>
> +<br>
> +template <typename ModelledPHI> struct DenseMapInfo {<br>
> + static inline ModelledPHI &getEmptyKey() {<br>
> + static ModelledPHI Dummy = ModelledPHI::createDummy(0);<br>
> + return Dummy;<br>
> + }<br>
> + static inline ModelledPHI &getTombstoneKey() {<br>
> + static ModelledPHI Dummy = ModelledPHI::createDummy(1);<br>
> + return Dummy;<br>
> + }<br>
> + static unsigned getHashValue(const ModelledPHI &V) { return V.hash(); }<br>
> + static bool isEqual(const ModelledPHI &LHS, const ModelledPHI &RHS) {<br>
> + return LHS == RHS;<br>
> + }<br>
> +};<br>
> +<br>
> +typedef DenseSet<ModelledPHI, DenseMapInfo<ModelledPHI>> ModelledPHISet;<br>
> +<br>
> +//===----------------------------------------------------------------------===//<br>
> +// ValueTable<br>
> +//===----------------------------------------------------------------------===//<br>
> +// This is a value number table where the value number is a function of the<br>
> +// *uses* of a value, rather than its operands. Thus, if VN(A) == VN(B) we know<br>
> +// that the program would be equivalent if we replaced A with PHI(A, B).<br>
> +//===----------------------------------------------------------------------===//<br>
> +<br>
> +/// A GVN expression describing how an instruction is used. The operands<br>
> +/// field of BasicExpression is used to store uses, not operands.<br>
> +///<br>
> +/// This class also contains fields for discriminators used when determining<br>
> +/// equivalence of instructions with sideeffects.<br>
> +class InstructionUseExpr : public GVNExpression::BasicExpression {<br>
> + unsigned MemoryUseOrder = -1;<br>
> + bool Volatile = false;<br>
> +<br>
> +public:<br>
> + InstructionUseExpr(Instruction *I, ArrayRecycler<Value *> &R,<br>
> + BumpPtrAllocator &A)<br>
> + : GVNExpression::BasicExpression(I->getNumUses()) {<br>
> + allocateOperands(R, A);<br>
> + setOpcode(I->getOpcode());<br>
> + setType(I->getType());<br>
> +<br>
> + for (auto &U : I->uses())<br>
> + op_push_back(U.getUser());<br>
> + std::sort(op_begin(), op_end());<br>
> + }<br>
> + void setMemoryUseOrder(unsigned MUO) { MemoryUseOrder = MUO; }<br>
> + void setVolatile(bool V) { Volatile = V; }<br>
> +<br>
> + virtual hash_code getHashValue() const {<br>
> + return hash_combine(GVNExpression::BasicExpression::getHashValue(),<br>
> + MemoryUseOrder, Volatile);<br>
> + }<br>
> +<br>
> + template <typename Function> hash_code getHashValue(Function MapFn) {<br>
> + hash_code H =<br>
> + hash_combine(getOpcode(), getType(), MemoryUseOrder, Volatile);<br>
> + for (auto *V : operands())<br>
> + H = hash_combine(H, MapFn(V));<br>
> + return H;<br>
> + }<br>
> +};<br>
> +<br>
> +class ValueTable {<br>
> + DenseMap<Value *, uint32_t> ValueNumbering;<br>
> + DenseMap<GVNExpression::Expression *, uint32_t> ExpressionNumbering;<br>
> + DenseMap<size_t, uint32_t> HashNumbering;<br>
> + BumpPtrAllocator Allocator;<br>
> + ArrayRecycler<Value *> Recycler;<br>
> + uint32_t nextValueNumber;<br>
> +<br>
> + /// Create an expression for I based on its opcode and its uses. If I<br>
> + /// touches or reads memory, the expression is also based upon its memory<br>
> + /// order - see \c getMemoryUseOrder().<br>
> + InstructionUseExpr *createExpr(Instruction *I) {<br>
> + InstructionUseExpr *E =<br>
> + new (Allocator) InstructionUseExpr(I, Recycler, Allocator);<br>
> + if (isMemoryInst(I))<br>
> + E->setMemoryUseOrder(getMemoryUseOrder(I));<br>
> +<br>
> + if (CmpInst *C = dyn_cast<CmpInst>(I)) {<br>
> + CmpInst::Predicate Predicate = C->getPredicate();<br>
> + E->setOpcode((C->getOpcode() << 8) | Predicate);<br>
> + }<br>
> + return E;<br>
> + }<br>
> +<br>
> + /// Helper to compute the value number for a memory instruction<br>
> + /// (LoadInst/StoreInst), including checking the memory ordering and<br>
> + /// volatility.<br>
> + template <class Inst> InstructionUseExpr *createMemoryExpr(Inst *I) {<br>
> + if (isStrongerThanUnordered(I->getOrdering()) || I->isAtomic())<br>
> + return nullptr;<br>
> + InstructionUseExpr *E = createExpr(I);<br>
> + E->setVolatile(I->isVolatile());<br>
> + return E;<br>
> + }<br>
> +<br>
> +public:<br>
> + /// Returns the value number for the specified value, assigning<br>
> + /// it a new number if it did not have one before.<br>
> + uint32_t lookupOrAdd(Value *V) {<br>
> + auto VI = ValueNumbering.find(V);<br>
> + if (VI != ValueNumbering.end())<br>
> + return VI->second;<br>
> +<br>
> + if (!isa<Instruction>(V)) {<br>
> + ValueNumbering[V] = nextValueNumber;<br>
> + return nextValueNumber++;<br>
> + }<br>
> +<br>
> + Instruction *I = cast<Instruction>(V);<br>
> + InstructionUseExpr *exp = nullptr;<br>
> + switch (I->getOpcode()) {<br>
> + case Instruction::Load:<br>
> + exp = createMemoryExpr(cast<LoadInst>(I));<br>
> + break;<br>
> + case Instruction::Store:<br>
> + exp = createMemoryExpr(cast<StoreInst>(I));<br>
> + break;<br>
> + case Instruction::Call:<br>
> + case Instruction::Invoke:<br>
> + case Instruction::Add:<br>
> + case Instruction::FAdd:<br>
> + case Instruction::Sub:<br>
> + case Instruction::FSub:<br>
> + case Instruction::Mul:<br>
> + case Instruction::FMul:<br>
> + case Instruction::UDiv:<br>
> + case Instruction::SDiv:<br>
> + case Instruction::FDiv:<br>
> + case Instruction::URem:<br>
> + case Instruction::SRem:<br>
> + case Instruction::FRem:<br>
> + case Instruction::Shl:<br>
> + case Instruction::LShr:<br>
> + case Instruction::AShr:<br>
> + case Instruction::And:<br>
> + case Instruction::Or:<br>
> + case Instruction::Xor:<br>
> + case Instruction::ICmp:<br>
> + case Instruction::FCmp:<br>
> + case Instruction::Trunc:<br>
> + case Instruction::ZExt:<br>
> + case Instruction::SExt:<br>
> + case Instruction::FPToUI:<br>
> + case Instruction::FPToSI:<br>
> + case Instruction::UIToFP:<br>
> + case Instruction::SIToFP:<br>
> + case Instruction::FPTrunc:<br>
> + case Instruction::FPExt:<br>
> + case Instruction::PtrToInt:<br>
> + case Instruction::IntToPtr:<br>
> + case Instruction::BitCast:<br>
> + case Instruction::Select:<br>
> + case Instruction::ExtractElement:<br>
> + case Instruction::InsertElement:<br>
> + case Instruction::ShuffleVector:<br>
> + case Instruction::InsertValue:<br>
> + case Instruction::GetElementPtr:<br>
> + exp = createExpr(I);<br>
> + break;<br>
> + default:<br>
> + break;<br>
> + }<br>
> +<br>
> + if (!exp) {<br>
> + ValueNumbering[V] = nextValueNumber;<br>
> + return nextValueNumber++;<br>
> + }<br>
> +<br>
> + uint32_t e = ExpressionNumbering[exp];<br>
> + if (!e) {<br>
> + hash_code H = exp->getHashValue([=](Value *V) { return lookupOrAdd(V); });<br>
> + auto I = HashNumbering.find(H);<br>
> + if (I != HashNumbering.end()) {<br>
> + e = I->second;<br>
> + } else {<br>
> + e = nextValueNumber++;<br>
> + HashNumbering[H] = e;<br>
> + ExpressionNumbering[exp] = e;<br>
> + }<br>
> + }<br>
> + ValueNumbering[V] = e;<br>
> + return e;<br>
> + }<br>
> +<br>
> + /// Returns the value number of the specified value. Fails if the value has<br>
> + /// not yet been numbered.<br>
> + uint32_t lookup(Value *V) const {<br>
> + auto VI = ValueNumbering.find(V);<br>
> + assert(VI != ValueNumbering.end() && "Value not numbered?");<br>
> + return VI->second;<br>
> + }<br>
> +<br>
> + /// Removes all value numberings and resets the value table.<br>
> + void clear() {<br>
> + ValueNumbering.clear();<br>
> + ExpressionNumbering.clear();<br>
> + HashNumbering.clear();<br>
> + Recycler.clear(Allocator);<br>
> + nextValueNumber = 1;<br>
> + }<br>
> +<br>
> + ValueTable() : nextValueNumber(1) {}<br>
> +<br>
> + /// \c Inst uses or touches memory. Return an ID describing the memory state<br>
> + /// at \c Inst such that if getMemoryUseOrder(I1) == getMemoryUseOrder(I2),<br>
> + /// the exact same memory operations happen after I1 and I2.<br>
> + ///<br>
> + /// This is a very hard problem in general, so we use domain-specific<br>
> + /// knowledge that we only ever check for equivalence between blocks sharing a<br>
> + /// single immediate successor that is common, and when determining if I1 ==<br>
> + /// I2 we will have already determined that next(I1) == next(I2). This<br>
> + /// inductive property allows us to simply return the value number of the next<br>
> + /// instruction that defines memory.<br>
> + uint32_t getMemoryUseOrder(Instruction *Inst) {<br>
> + auto *BB = Inst->getParent();<br>
> + for (auto I = std::next(Inst->getIterator()), E = BB->end();<br>
> + I != E && !I->isTerminator(); ++I) {<br>
> + if (!isMemoryInst(&*I))<br>
> + continue;<br>
> + if (isa<LoadInst>(&*I))<br>
> + continue;<br>
> + CallInst *CI = dyn_cast<CallInst>(&*I);<br>
> + if (CI && CI->onlyReadsMemory())<br>
> + continue;<br>
> + InvokeInst *II = dyn_cast<InvokeInst>(&*I);<br>
> + if (II && II->onlyReadsMemory())<br>
> + continue;<br>
> + return lookupOrAdd(&*I);<br>
> + }<br>
> + return 0;<br>
> + }<br>
> +};<br>
> +<br>
> +//===----------------------------------------------------------------------===//<br>
> +<br>
> +class GVNSink {<br>
> +public:<br>
> + GVNSink() : VN() {}<br>
> + bool run(Function &F) {<br>
> + DEBUG(dbgs() << "GVNSink: running on function @" << F.getName() << "\n");<br>
> +<br>
> + unsigned NumSunk = 0;<br>
> + ReversePostOrderTraversal<Function*> RPOT(&F);<br>
> + for (auto *N : RPOT)<br>
> + NumSunk += sinkBB(N);<br>
> +<br>
> + return NumSunk > 0;<br>
> + }<br>
> +<br>
> +private:<br>
> + ValueTable VN;<br>
> +<br>
> + bool isInstructionBlacklisted(Instruction *I) {<br>
> + // These instructions may change or break semantics if moved.<br>
> + if (isa<PHINode>(I) || I->isEHPad() || isa<AllocaInst>(I) ||<br>
> + I->getType()->isTokenTy())<br>
> + return true;<br>
> + return false;<br>
> + }<br>
> +<br>
> + /// The main heuristic function. Analyze the set of instructions pointed to by<br>
> + /// LRI and return a candidate solution if these instructions can be sunk, or<br>
> + /// None otherwise.<br>
> + Optional<SinkingInstructionCandidate> analyzeInstructionForSinking(<br>
> + LockstepReverseIterator &LRI, unsigned &InstNum, unsigned &MemoryInstNum,<br>
> + ModelledPHISet &NeededPHIs, SmallPtrSetImpl<Value *> &PHIContents);<br>
> +<br>
> + /// Create a ModelledPHI for each PHI in BB, adding to PHIs.<br>
> + void analyzeInitialPHIs(BasicBlock *BB, ModelledPHISet &PHIs,<br>
> + SmallPtrSetImpl<Value *> &PHIContents) {<br>
> + for (auto &I : *BB) {<br>
> + auto *PN = dyn_cast<PHINode>(&I);<br>
> + if (!PN)<br>
> + return;<br>
> +<br>
> + auto MPHI = ModelledPHI(PN);<br>
> + PHIs.insert(MPHI);<br>
> + for (auto *V : MPHI.getValues())<br>
> + PHIContents.insert(V);<br>
> + }<br>
> + }<br>
> +<br>
> + /// The main instruction sinking driver. Set up state and try and sink<br>
> + /// instructions into BBEnd from its predecessors.<br>
> + unsigned sinkBB(BasicBlock *BBEnd);<br>
> +<br>
> + /// Perform the actual mechanics of sinking an instruction from Blocks into<br>
> + /// BBEnd, which is their only successor.<br>
> + void sinkLastInstruction(ArrayRef<BasicBlock *> Blocks, BasicBlock *BBEnd);<br>
> +<br>
> + /// Remove PHIs that all have the same incoming value.<br>
> + void foldPointlessPHINodes(BasicBlock *BB) {<br>
> + auto I = BB->begin();<br>
> + while (PHINode *PN = dyn_cast<PHINode>(I++)) {<br>
> + if (!all_of(PN->incoming_values(),<br>
> + [&](const Value *V) { return V == PN->getIncomingValue(0); }))<br>
> + continue;<br>
> + if (PN->getIncomingValue(0) != PN)<br>
> + PN->replaceAllUsesWith(PN->getIncomingValue(0));<br>
> + else<br>
> + PN->replaceAllUsesWith(UndefValue::get(PN->getType()));<br>
> + PN->eraseFromParent();<br>
> + }<br>
> + }<br>
> +};<br>
> +<br>
> +Optional<SinkingInstructionCandidate> GVNSink::analyzeInstructionForSinking(<br>
> + LockstepReverseIterator &LRI, unsigned &InstNum, unsigned &MemoryInstNum,<br>
> + ModelledPHISet &NeededPHIs, SmallPtrSetImpl<Value *> &PHIContents) {<br>
> + auto Insts = *LRI;<br>
> + DEBUG(dbgs() << " -- Analyzing instruction set: [\n"; for (auto *I<br>
> + : Insts) {<br>
> + I->dump();<br>
> + } dbgs() << " ]\n";);<br>
> +<br>
> + DenseMap<uint32_t, unsigned> VNums;<br>
> + for (auto *I : Insts) {<br>
> + uint32_t N = VN.lookupOrAdd(I);<br>
> + DEBUG(dbgs() << " VN=" << utohexstr(N) << " for" << *I << "\n");<br>
> + if (N == ~0U)<br>
> + return None;<br>
> + VNums[N]++;<br>
> + }<br>
> + unsigned VNumToSink =<br>
> + std::max_element(VNums.begin(), VNums.end(),<br>
> + [](const std::pair<uint32_t, unsigned> &I,<br>
> + const std::pair<uint32_t, unsigned> &J) {<br>
> + return I.second < J.second;<br>
> + })<br>
> + ->first;<br>
> +<br>
> + if (VNums[VNumToSink] == 1)<br>
> + // Can't sink anything!<br>
> + return None;<br>
> +<br>
> + // Now restrict the number of incoming blocks down to only those with<br>
> + // VNumToSink.<br>
> + auto &ActivePreds = LRI.getActiveBlocks();<br>
> + unsigned InitialActivePredSize = ActivePreds.size();<br>
> + SmallVector<Instruction *, 4> NewInsts;<br>
> + for (auto *I : Insts) {<br>
> + if (VN.lookup(I) != VNumToSink)<br>
> + ActivePreds.erase(I->getParent());<br>
> + else<br>
> + NewInsts.push_back(I);<br>
> + }<br>
> + for (auto *I : NewInsts)<br>
> + if (isInstructionBlacklisted(I))<br>
> + return None;<br>
> +<br>
> + // If we've restricted the incoming blocks, restrict all needed PHIs also<br>
> + // to that set.<br>
> + bool RecomputePHIContents = false;<br>
> + if (ActivePreds.size() != InitialActivePredSize) {<br>
> + ModelledPHISet NewNeededPHIs;<br>
> + for (auto P : NeededPHIs) {<br>
> + P.restrictToBlocks(ActivePreds);<br>
> + NewNeededPHIs.insert(P);<br>
> + }<br>
> + NeededPHIs = NewNeededPHIs;<br>
> + LRI.restrictToBlocks(ActivePreds);<br>
> + RecomputePHIContents = true;<br>
> + }<br>
> +<br>
> + // The sunk instruction's results.<br>
> + ModelledPHI NewPHI(NewInsts, ActivePreds);<br>
> +<br>
> + // Does sinking this instruction render previous PHIs redundant?<br>
> + if (NeededPHIs.find(NewPHI) != NeededPHIs.end()) {<br>
> + NeededPHIs.erase(NewPHI);<br>
> + RecomputePHIContents = true;<br>
> + }<br>
> +<br>
> + if (RecomputePHIContents) {<br>
> + // The needed PHIs have changed, so recompute the set of all needed<br>
> + // values.<br>
> + PHIContents.clear();<br>
> + for (auto &PHI : NeededPHIs)<br>
> + PHIContents.insert(PHI.getValues().begin(), PHI.getValues().end());<br>
> + }<br>
> +<br>
> + // Is this instruction required by a later PHI that doesn't match this PHI?<br>
> + // if so, we can't sink this instruction.<br>
> + for (auto *V : NewPHI.getValues())<br>
> + if (PHIContents.count(V))<br>
> + // V exists in this PHI, but the whole PHI is different to NewPHI<br>
> + // (else it would have been removed earlier). We cannot continue<br>
> + // because this isn't representable.<br>
> + return None;<br>
> +<br>
> + // Which operands need PHIs?<br>
> + // FIXME: If any of these fail, we should partition up the candidates to<br>
> + // try and continue making progress.<br>
> + Instruction *I0 = NewInsts[0];<br>
> + for (unsigned OpNum = 0, E = I0->getNumOperands(); OpNum != E; ++OpNum) {<br>
> + ModelledPHI PHI(NewInsts, OpNum, ActivePreds);<br>
> + if (PHI.areAllIncomingValuesSame())<br>
> + continue;<br>
> + if (!canReplaceOperandWithVariable(I0, OpNum))<br>
> + // We can 't create a PHI from this instruction!<br>
> + return None;<br>
> + if (NeededPHIs.count(PHI))<br>
> + continue;<br>
> + if (!PHI.areAllIncomingValuesSameType())<br>
> + return None;<br>
> + // Don't create indirect calls! The called value is the final operand.<br>
> + if ((isa<CallInst>(I0) || isa<InvokeInst>(I0)) && OpNum == E - 1 &&<br>
> + PHI.areAnyIncomingValuesConstant())<br>
> + return None;<br>
> +<br>
> + NeededPHIs.reserve(NeededPHIs.size());<br>
> + NeededPHIs.insert(PHI);<br>
> + PHIContents.insert(PHI.getValues().begin(), PHI.getValues().end());<br>
> + }<br>
> +<br>
> + if (isMemoryInst(NewInsts[0]))<br>
> + ++MemoryInstNum;<br>
> +<br>
> + SinkingInstructionCandidate Cand;<br>
> + Cand.NumInstructions = ++InstNum;<br>
> + Cand.NumMemoryInsts = MemoryInstNum;<br>
> + Cand.NumBlocks = ActivePreds.size();<br>
> + Cand.NumPHIs = NeededPHIs.size();<br>
> + for (auto *C : ActivePreds)<br>
> + Cand.Blocks.push_back(C);<br>
> +<br>
> + return Cand;<br>
> +}<br>
> +<br>
> +unsigned GVNSink::sinkBB(BasicBlock *BBEnd) {<br>
> + DEBUG(dbgs() << "GVNSink: running on basic block ";<br>
> + BBEnd->printAsOperand(dbgs()); dbgs() << "\n");<br>
> + SmallVector<BasicBlock *, 4> Preds;<br>
> + for (auto *B : predecessors(BBEnd)) {<br>
> + auto *T = B->getTerminator();<br>
> + if (isa<BranchInst>(T) || isa<SwitchInst>(T))<br>
> + Preds.push_back(B);<br>
> + else<br>
> + return 0;<br>
> + }<br>
> + if (Preds.size() < 2)<br>
> + return 0;<br>
> + std::sort(Preds.begin(), Preds.end());<br>
> +<br>
> + unsigned NumOrigPreds = Preds.size();<br>
> + // We can only sink instructions through unconditional branches.<br>
> + for (auto I = Preds.begin(); I != Preds.end();) {<br>
> + if ((*I)->getTerminator()->getNumSuccessors() != 1)<br>
> + I = Preds.erase(I);<br>
> + else<br>
> + ++I;<br>
> + }<br>
> +<br>
> + LockstepReverseIterator LRI(Preds);<br>
> + SmallVector<SinkingInstructionCandidate, 4> Candidates;<br>
> + unsigned InstNum = 0, MemoryInstNum = 0;<br>
> + ModelledPHISet NeededPHIs;<br>
> + SmallPtrSet<Value *, 4> PHIContents;<br>
> + analyzeInitialPHIs(BBEnd, NeededPHIs, PHIContents);<br>
> + unsigned NumOrigPHIs = NeededPHIs.size();<br>
> +<br>
> + while (LRI.isValid()) {<br>
> + auto Cand = analyzeInstructionForSinking(LRI, InstNum, MemoryInstNum,<br>
> + NeededPHIs, PHIContents);<br>
> + if (!Cand)<br>
> + break;<br>
> + Cand->calculateCost(NumOrigPHIs, Preds.size());<br>
> + Candidates.emplace_back(*Cand);<br>
> + --LRI;<br>
> + }<br>
> +<br>
> + std::stable_sort(<br>
> + Candidates.begin(), Candidates.end(),<br>
> + [](const SinkingInstructionCandidate &A,<br>
> + const SinkingInstructionCandidate &B) { return A >= B; });<br>
<br>
std::stable_sort requires a strict weak ordering, so the predicate<br>
should never return true if A and B are equal. This code violates<br>
that.<br>
<br>
> + DEBUG(dbgs() << " -- Sinking candidates:\n"; for (auto &C<br>
> + : Candidates) dbgs()<br>
> + << " " << C << "\n";);<br>
> +<br>
> + // Pick the top candidate, as long it is positive!<br>
> + if (Candidates.empty() || Candidates.front().Cost <= 0)<br>
> + return 0;<br>
> + auto C = Candidates.front();<br>
> +<br>
> + DEBUG(dbgs() << " -- Sinking: " << C << "\n");<br>
> + BasicBlock *InsertBB = BBEnd;<br>
> + if (C.Blocks.size() < NumOrigPreds) {<br>
> + DEBUG(dbgs() << " -- Splitting edge to "; BBEnd->printAsOperand(dbgs());<br>
> + dbgs() << "\n");<br>
> + InsertBB = SplitBlockPredecessors(BBEnd, C.Blocks, ".gvnsink.split");<br>
> + if (!InsertBB) {<br>
> + DEBUG(dbgs() << " -- FAILED to split edge!\n");<br>
> + // Edge couldn't be split.<br>
> + return 0;<br>
> + }<br>
> + }<br>
> +<br>
> + for (unsigned I = 0; I < C.NumInstructions; ++I)<br>
> + sinkLastInstruction(C.Blocks, InsertBB);<br>
> +<br>
> + return C.NumInstructions;<br>
> +}<br>
> +<br>
> +void GVNSink::sinkLastInstruction(ArrayRef<BasicBlock *> Blocks,<br>
> + BasicBlock *BBEnd) {<br>
> + SmallVector<Instruction *, 4> Insts;<br>
> + for (BasicBlock *BB : Blocks)<br>
> + Insts.push_back(BB->getTerminator()->getPrevNode());<br>
> + Instruction *I0 = Insts.front();<br>
> +<br>
> + SmallVector<Value *, 4> NewOperands;<br>
> + for (unsigned O = 0, E = I0->getNumOperands(); O != E; ++O) {<br>
> + bool NeedPHI = any_of(Insts, [&I0, O](const Instruction *I) {<br>
> + return I->getOperand(O) != I0->getOperand(O);<br>
> + });<br>
> + if (!NeedPHI) {<br>
> + NewOperands.push_back(I0->getOperand(O));<br>
> + continue;<br>
> + }<br>
> +<br>
> + // Create a new PHI in the successor block and populate it.<br>
> + auto *Op = I0->getOperand(O);<br>
> + assert(!Op->getType()->isTokenTy() && "Can't PHI tokens!");<br>
> + auto *PN = PHINode::Create(Op->getType(), Insts.size(),<br>
> + Op->getName() + ".sink", &BBEnd->front());<br>
> + for (auto *I : Insts)<br>
> + PN->addIncoming(I->getOperand(O), I->getParent());<br>
> + NewOperands.push_back(PN);<br>
> + }<br>
> +<br>
> + // Arbitrarily use I0 as the new "common" instruction; remap its operands<br>
> + // and move it to the start of the successor block.<br>
> + for (unsigned O = 0, E = I0->getNumOperands(); O != E; ++O)<br>
> + I0->getOperandUse(O).set(NewOperands[O]);<br>
> + I0->moveBefore(&*BBEnd->getFirstInsertionPt());<br>
> +<br>
> + // Update metadata and IR flags.<br>
> + for (auto *I : Insts)<br>
> + if (I != I0) {<br>
> + combineMetadataForCSE(I0, I);<br>
> + I0->andIRFlags(I);<br>
> + }<br>
> +<br>
> + for (auto *I : Insts)<br>
> + if (I != I0)<br>
> + I->replaceAllUsesWith(I0);<br>
> + foldPointlessPHINodes(BBEnd);<br>
> +<br>
> + // Finally nuke all instructions apart from the common instruction.<br>
> + for (auto *I : Insts)<br>
> + if (I != I0)<br>
> + I->eraseFromParent();<br>
> +<br>
> + NumRemoved += Insts.size() - 1;<br>
> +}<br>
> +<br>
> +////////////////////////////////////////////////////////////////////////////////<br>
> +// Pass machinery / boilerplate<br>
> +<br>
> +class GVNSinkLegacyPass : public FunctionPass {<br>
> +public:<br>
> + static char ID;<br>
> +<br>
> + GVNSinkLegacyPass() : FunctionPass(ID) {<br>
> + initializeGVNSinkLegacyPassPass(*PassRegistry::getPassRegistry());<br>
> + }<br>
> +<br>
> + bool runOnFunction(Function &F) override {<br>
> + if (skipFunction(F))<br>
> + return false;<br>
> + GVNSink G;<br>
> + return G.run(F);<br>
> + }<br>
> +<br>
> + void getAnalysisUsage(AnalysisUsage &AU) const override {<br>
> + AU.addPreserved<GlobalsAAWrapperPass>();<br>
> + }<br>
> +};<br>
> +} // namespace<br>
> +<br>
> +PreservedAnalyses GVNSinkPass::run(Function &F, FunctionAnalysisManager &AM) {<br>
> + GVNSink G;<br>
> + if (!G.run(F))<br>
> + return PreservedAnalyses::all();<br>
> +<br>
> + PreservedAnalyses PA;<br>
> + PA.preserve<GlobalsAA>();<br>
> + return PA;<br>
> +}<br>
> +<br>
> +char GVNSinkLegacyPass::ID = 0;<br>
> +INITIALIZE_PASS_BEGIN(GVNSinkLegacyPass, "gvn-sink",<br>
> + "Early GVN sinking of Expressions", false, false)<br>
> +INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)<br>
> +INITIALIZE_PASS_DEPENDENCY(PostDominatorTreeWrapperPass)<br>
> +INITIALIZE_PASS_END(GVNSinkLegacyPass, "gvn-sink",<br>
> + "Early GVN sinking of Expressions", false, false)<br>
> +<br>
> +FunctionPass *llvm::createGVNSinkPass() { return new GVNSinkLegacyPass(); }<br>
><br>
> Modified: llvm/trunk/lib/Transforms/Scalar/Scalar.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/Scalar.cpp?rev=303850&r1=303849&r2=303850&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/Scalar.cpp?rev=303850&r1=303849&r2=303850&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/Transforms/Scalar/Scalar.cpp (original)<br>
> +++ llvm/trunk/lib/Transforms/Scalar/Scalar.cpp Thu May 25 07:51:11 2017<br>
> @@ -48,6 +48,7 @@ void llvm::initializeScalarOpts(PassRegi<br>
> initializeEarlyCSELegacyPassPass(Registry);<br>
> initializeEarlyCSEMemSSALegacyPassPass(Registry);<br>
> initializeGVNHoistLegacyPassPass(Registry);<br>
> + initializeGVNSinkLegacyPassPass(Registry);<br>
> initializeFlattenCFGPassPass(Registry);<br>
> initializeInductiveRangeCheckEliminationPass(Registry);<br>
> initializeIndVarSimplifyLegacyPassPass(Registry);<br>
><br>
> Modified: llvm/trunk/lib/Transforms/Utils/Local.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Utils/Local.cpp?rev=303850&r1=303849&r2=303850&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Utils/Local.cpp?rev=303850&r1=303849&r2=303850&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/Transforms/Utils/Local.cpp (original)<br>
> +++ llvm/trunk/lib/Transforms/Utils/Local.cpp Thu May 25 07:51:11 2017<br>
> @@ -2109,3 +2109,48 @@ void llvm::maybeMarkSanitizerLibraryCall<br>
> !F->doesNotAccessMemory())<br>
> CI->addAttribute(AttributeList::FunctionIndex, Attribute::NoBuiltin);<br>
> }<br>
> +<br>
> +bool llvm::canReplaceOperandWithVariable(const Instruction *I, unsigned OpIdx) {<br>
> + // We can't have a PHI with a metadata type.<br>
> + if (I->getOperand(OpIdx)->getType()->isMetadataTy())<br>
> + return false;<br>
> +<br>
> + // Early exit.<br>
> + if (!isa<Constant>(I->getOperand(OpIdx)))<br>
> + return true;<br>
> +<br>
> + switch (I->getOpcode()) {<br>
> + default:<br>
> + return true;<br>
> + case Instruction::Call:<br>
> + case Instruction::Invoke:<br>
> + // Many arithmetic intrinsics have no issue taking a<br>
> + // variable, however it's hard to distingish these from<br>
> + // specials such as @llvm.frameaddress that require a constant.<br>
> + if (isa<IntrinsicInst>(I))<br>
> + return false;<br>
> +<br>
> + // Constant bundle operands may need to retain their constant-ness for<br>
> + // correctness.<br>
> + if (ImmutableCallSite(I).isBundleOperand(OpIdx))<br>
> + return false;<br>
> + return true;<br>
> + case Instruction::ShuffleVector:<br>
> + // Shufflevector masks are constant.<br>
> + return OpIdx != 2;<br>
> + case Instruction::ExtractValue:<br>
> + case Instruction::InsertValue:<br>
> + // All operands apart from the first are constant.<br>
> + return OpIdx == 0;<br>
> + case Instruction::Alloca:<br>
> + return false;<br>
> + case Instruction::GetElementPtr:<br>
> + if (OpIdx == 0)<br>
> + return true;<br>
> + gep_type_iterator It = gep_type_begin(I);<br>
> + for (auto E = std::next(It, OpIdx); It != E; ++It)<br>
> + if (It.isStruct())<br>
> + return false;<br>
> + return true;<br>
> + }<br>
> +}<br>
><br>
> Modified: llvm/trunk/lib/Transforms/Utils/SimplifyCFG.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Utils/SimplifyCFG.cpp?rev=303850&r1=303849&r2=303850&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Utils/SimplifyCFG.cpp?rev=303850&r1=303849&r2=303850&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/Transforms/Utils/SimplifyCFG.cpp (original)<br>
> +++ llvm/trunk/lib/Transforms/Utils/SimplifyCFG.cpp Thu May 25 07:51:11 2017<br>
> @@ -1376,53 +1376,6 @@ HoistTerminator:<br>
> return true;<br>
> }<br>
><br>
> -// Is it legal to place a variable in operand \c OpIdx of \c I?<br>
> -// FIXME: This should be promoted to Instruction.<br>
> -static bool canReplaceOperandWithVariable(const Instruction *I,<br>
> - unsigned OpIdx) {<br>
> - // We can't have a PHI with a metadata type.<br>
> - if (I->getOperand(OpIdx)->getType()->isMetadataTy())<br>
> - return false;<br>
> -<br>
> - // Early exit.<br>
> - if (!isa<Constant>(I->getOperand(OpIdx)))<br>
> - return true;<br>
> -<br>
> - switch (I->getOpcode()) {<br>
> - default:<br>
> - return true;<br>
> - case Instruction::Call:<br>
> - case Instruction::Invoke:<br>
> - // FIXME: many arithmetic intrinsics have no issue taking a<br>
> - // variable, however it's hard to distingish these from<br>
> - // specials such as @llvm.frameaddress that require a constant.<br>
> - if (isa<IntrinsicInst>(I))<br>
> - return false;<br>
> -<br>
> - // Constant bundle operands may need to retain their constant-ness for<br>
> - // correctness.<br>
> - if (ImmutableCallSite(I).isBundleOperand(OpIdx))<br>
> - return false;<br>
> -<br>
> - return true;<br>
> -<br>
> - case Instruction::ShuffleVector:<br>
> - // Shufflevector masks are constant.<br>
> - return OpIdx != 2;<br>
> - case Instruction::ExtractValue:<br>
> - case Instruction::InsertValue:<br>
> - // All operands apart from the first are constant.<br>
> - return OpIdx == 0;<br>
> - case Instruction::Alloca:<br>
> - return false;<br>
> - case Instruction::GetElementPtr:<br>
> - if (OpIdx == 0)<br>
> - return true;<br>
> - gep_type_iterator It = std::next(gep_type_begin(I), OpIdx - 1);<br>
> - return It.isSequential();<br>
> - }<br>
> -}<br>
> -<br>
> // All instructions in Insts belong to different blocks that all unconditionally<br>
> // branch to a common successor. Analyze each instruction and return true if it<br>
> // would be possible to sink them into their successor, creating one common<br>
><br>
> Added: llvm/trunk/test/Transforms/GVNSink/dither.ll<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/GVNSink/dither.ll?rev=303850&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/GVNSink/dither.ll?rev=303850&view=auto</a><br>
> ==============================================================================<br>
> --- llvm/trunk/test/Transforms/GVNSink/dither.ll (added)<br>
> +++ llvm/trunk/test/Transforms/GVNSink/dither.ll Thu May 25 07:51:11 2017<br>
> @@ -0,0 +1,42 @@<br>
> +; RUN: opt < %s -S -gvn-sink | FileCheck %s<br>
> +<br>
> +; Because %tmp17 has flipped operands to its equivalents %tmp14 and %tmp7, we<br>
> +; can't sink the zext as we'd need a shuffling PHI in between.<br>
> +;<br>
> +; Just sinking the zext isn't profitable, so ensure nothing is sunk.<br>
> +<br>
> +; CHECK-LABEL: @hoge<br>
> +; CHECK-NOT: bb18.gvnsink.split<br>
> +define void @hoge() {<br>
> +bb:<br>
> + br i1 undef, label %bb4, label %bb11<br>
> +<br>
> +bb4: ; preds = %bb3<br>
> + br i1 undef, label %bb6, label %bb8<br>
> +<br>
> +bb6: ; preds = %bb5<br>
> + %tmp = zext i16 undef to i64<br>
> + %tmp7 = add i64 %tmp, undef<br>
> + br label %bb18<br>
> +<br>
> +bb8: ; preds = %bb5<br>
> + %tmp9 = zext i16 undef to i64<br>
> + br label %bb18<br>
> +<br>
> +bb11: ; preds = %bb10<br>
> + br i1 undef, label %bb12, label %bb15<br>
> +<br>
> +bb12: ; preds = %bb11<br>
> + %tmp13 = zext i16 undef to i64<br>
> + %tmp14 = add i64 %tmp13, undef<br>
> + br label %bb18<br>
> +<br>
> +bb15: ; preds = %bb11<br>
> + %tmp16 = zext i16 undef to i64<br>
> + %tmp17 = add i64 undef, %tmp16<br>
> + br label %bb18<br>
> +<br>
> +bb18: ; preds = %bb15, %bb12, %bb8, %bb6<br>
> + %tmp19 = phi i64 [ %tmp7, %bb6 ], [ undef, %bb8 ], [ %tmp14, %bb12 ], [ %tmp17, %bb15 ]<br>
> + unreachable<br>
> +}<br>
><br>
> Added: llvm/trunk/test/Transforms/GVNSink/indirect-call.ll<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/GVNSink/indirect-call.ll?rev=303850&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/GVNSink/indirect-call.ll?rev=303850&view=auto</a><br>
> ==============================================================================<br>
> --- llvm/trunk/test/Transforms/GVNSink/indirect-call.ll (added)<br>
> +++ llvm/trunk/test/Transforms/GVNSink/indirect-call.ll Thu May 25 07:51:11 2017<br>
> @@ -0,0 +1,70 @@<br>
> +; RUN: opt < %s -gvn-sink -simplifycfg -simplifycfg-sink-common=false -S | FileCheck %s<br>
> +<br>
> +declare i8 @ext(i1)<br>
> +<br>
> +define zeroext i1 @test1(i1 zeroext %flag, i32 %blksA, i32 %blksB, i32 %nblks, i8(i1)* %ext) {<br>
> +entry:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +; CHECK-LABEL: test1<br>
> +; CHECK: call i8 @ext<br>
> +; CHECK: call i8 %ext<br>
> +if.then:<br>
> + %frombool1 = call i8 @ext(i1 %cmp)<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %frombool3 = call i8 %ext(i1 %cmp)<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %obeys.0 = phi i8 [ %frombool1, %if.then ], [ %frombool3, %if.else ]<br>
> + %tobool4 = icmp ne i8 %obeys.0, 0<br>
> + ret i1 %tobool4<br>
> +}<br>
> +<br>
> +define zeroext i1 @test2(i1 zeroext %flag, i32 %blksA, i32 %blksB, i32 %nblks, i8(i1)* %ext) {<br>
> +entry:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +; CHECK-LABEL: test2<br>
> +; CHECK: call i8 %ext<br>
> +; CHECK-NOT: call<br>
> +if.then:<br>
> + %frombool1 = call i8 %ext(i1 %cmp)<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %frombool3 = call i8 %ext(i1 %cmp)<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %obeys.0 = phi i8 [ %frombool1, %if.then ], [ %frombool3, %if.else ]<br>
> + %tobool4 = icmp ne i8 %obeys.0, 0<br>
> + ret i1 %tobool4<br>
> +}<br>
> +<br>
> +define zeroext i1 @test3(i1 zeroext %flag, i32 %blksA, i32 %blksB, i32 %nblks, i8(i1)* %ext1, i8(i1)* %ext2) {<br>
> +entry:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +; CHECK-LABEL: test3<br>
> +; CHECK: %[[x:.*]] = select i1 %flag, i8 (i1)* %ext1, i8 (i1)* %ext2<br>
> +; CHECK: call i8 %[[x]](i1 %cmp)<br>
> +; CHECK-NOT: call<br>
> +if.then:<br>
> + %frombool1 = call i8 %ext1(i1 %cmp)<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %frombool3 = call i8 %ext2(i1 %cmp)<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %obeys.0 = phi i8 [ %frombool1, %if.then ], [ %frombool3, %if.else ]<br>
> + %tobool4 = icmp ne i8 %obeys.0, 0<br>
> + ret i1 %tobool4<br>
> +}<br>
><br>
> Added: llvm/trunk/test/Transforms/GVNSink/sink-common-code.ll<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/GVNSink/sink-common-code.ll?rev=303850&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/GVNSink/sink-common-code.ll?rev=303850&view=auto</a><br>
> ==============================================================================<br>
> --- llvm/trunk/test/Transforms/GVNSink/sink-common-code.ll (added)<br>
> +++ llvm/trunk/test/Transforms/GVNSink/sink-common-code.ll Thu May 25 07:51:11 2017<br>
> @@ -0,0 +1,694 @@<br>
> +; RUN: opt < %s -gvn-sink -simplifycfg -simplifycfg-sink-common=false -S | FileCheck %s<br>
> +<br>
> +define zeroext i1 @test1(i1 zeroext %flag, i32 %blksA, i32 %blksB, i32 %nblks) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +; CHECK-LABEL: test1<br>
> +; CHECK: add<br>
> +; CHECK: select<br>
> +; CHECK: icmp<br>
> +; CHECK-NOT: br<br>
> +if.then:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + %frombool1 = zext i1 %cmp to i8<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %add = add i32 %nblks, %blksB<br>
> + %cmp2 = icmp ule i32 %add, %blksA<br>
> + %frombool3 = zext i1 %cmp2 to i8<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %obeys.0 = phi i8 [ %frombool1, %if.then ], [ %frombool3, %if.else ]<br>
> + %tobool4 = icmp ne i8 %obeys.0, 0<br>
> + ret i1 %tobool4<br>
> +}<br>
> +<br>
> +define zeroext i1 @test2(i1 zeroext %flag, i32 %blksA, i32 %blksB, i32 %nblks) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +; CHECK-LABEL: test2<br>
> +; CHECK: add<br>
> +; CHECK: select<br>
> +; CHECK: icmp<br>
> +; CHECK-NOT: br<br>
> +if.then:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + %frombool1 = zext i1 %cmp to i8<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %add = add i32 %nblks, %blksB<br>
> + %cmp2 = icmp uge i32 %blksA, %add<br>
> + %frombool3 = zext i1 %cmp2 to i8<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %obeys.0 = phi i8 [ %frombool1, %if.then ], [ %frombool3, %if.else ]<br>
> + %tobool4 = icmp ne i8 %obeys.0, 0<br>
> + ret i1 %tobool4<br>
> +}<br>
> +<br>
> +declare i32 @foo(i32, i32) nounwind readnone<br>
> +<br>
> +define i32 @test3(i1 zeroext %flag, i32 %x, i32 %y) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %x0 = call i32 @foo(i32 %x, i32 0) nounwind readnone<br>
> + %y0 = call i32 @foo(i32 %x, i32 1) nounwind readnone<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %x1 = call i32 @foo(i32 %y, i32 0) nounwind readnone<br>
> + %y1 = call i32 @foo(i32 %y, i32 1) nounwind readnone<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %xx = phi i32 [ %x0, %if.then ], [ %x1, %if.else ]<br>
> + %yy = phi i32 [ %y0, %if.then ], [ %y1, %if.else ]<br>
> + %ret = add i32 %xx, %yy<br>
> + ret i32 %ret<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test3<br>
> +; CHECK: select<br>
> +; CHECK: call<br>
> +; CHECK: call<br>
> +; CHECK: add<br>
> +; CHECK-NOT: br<br>
> +<br>
> +define i32 @test4(i1 zeroext %flag, i32 %x, i32* %y) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %a = add i32 %x, 5<br>
> + store i32 %a, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %b = add i32 %x, 7<br>
> + store i32 %b, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test4<br>
> +; CHECK: select<br>
> +; CHECK: store<br>
> +; CHECK-NOT: store<br>
> +<br>
> +define i32 @test5(i1 zeroext %flag, i32 %x, i32* %y) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %a = add i32 %x, 5<br>
> + store volatile i32 %a, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %b = add i32 %x, 7<br>
> + store i32 %b, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test5<br>
> +; CHECK: store volatile<br>
> +; CHECK: store<br>
> +<br>
> +define i32 @test6(i1 zeroext %flag, i32 %x, i32* %y) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %a = add i32 %x, 5<br>
> + store volatile i32 %a, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %b = add i32 %x, 7<br>
> + store volatile i32 %b, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test6<br>
> +; CHECK: select<br>
> +; CHECK: store volatile<br>
> +; CHECK-NOT: store<br>
> +<br>
> +define i32 @test7(i1 zeroext %flag, i32 %x, i32* %y) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %z = load volatile i32, i32* %y<br>
> + %a = add i32 %z, 5<br>
> + store volatile i32 %a, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %w = load volatile i32, i32* %y<br>
> + %b = add i32 %w, 7<br>
> + store volatile i32 %b, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test7<br>
> +; CHECK-DAG: select<br>
> +; CHECK-DAG: load volatile<br>
> +; CHECK: store volatile<br>
> +; CHECK-NOT: load<br>
> +; CHECK-NOT: store<br>
> +<br>
> +; The extra store in %if.then means %z and %w are not equivalent.<br>
> +define i32 @test9(i1 zeroext %flag, i32 %x, i32* %y, i32* %p) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + store i32 7, i32* %p<br>
> + %z = load volatile i32, i32* %y<br>
> + store i32 6, i32* %p<br>
> + %a = add i32 %z, 5<br>
> + store volatile i32 %a, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %w = load volatile i32, i32* %y<br>
> + %b = add i32 %w, 7<br>
> + store volatile i32 %b, i32* %y<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test9<br>
> +; CHECK: add<br>
> +; CHECK: add<br>
> +<br>
> +%struct.anon = type { i32, i32 }<br>
> +<br>
> +; The GEP indexes a struct type so cannot have a variable last index.<br>
> +define i32 @test10(i1 zeroext %flag, i32 %x, i32* %y, %struct.anon* %s) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %dummy = add i32 %x, 5<br>
> + %gepa = getelementptr inbounds %struct.anon, %struct.anon* %s, i32 0, i32 0<br>
> + store volatile i32 %x, i32* %gepa<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %dummy1 = add i32 %x, 6<br>
> + %gepb = getelementptr inbounds %struct.anon, %struct.anon* %s, i32 0, i32 1<br>
> + store volatile i32 %x, i32* %gepb<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test10<br>
> +; CHECK: getelementptr<br>
> +; CHECK: store volatile<br>
> +; CHECK: getelementptr<br>
> +; CHECK: store volatile<br>
> +<br>
> +; The shufflevector's mask operand cannot be merged in a PHI.<br>
> +define i32 @test11(i1 zeroext %flag, i32 %w, <2 x i32> %x, <2 x i32> %y) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %dummy = add i32 %w, 5<br>
> + %sv1 = shufflevector <2 x i32> %x, <2 x i32> %y, <2 x i32> <i32 0, i32 1><br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %dummy1 = add i32 %w, 6<br>
> + %sv2 = shufflevector <2 x i32> %x, <2 x i32> %y, <2 x i32> <i32 1, i32 0><br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %p = phi <2 x i32> [ %sv1, %if.then ], [ %sv2, %if.else ]<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test11<br>
> +; CHECK: shufflevector<br>
> +; CHECK: shufflevector<br>
> +<br>
> +; We can't common an intrinsic!<br>
> +define i32 @test12(i1 zeroext %flag, i32 %w, i32 %x, i32 %y) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %dummy = add i32 %w, 5<br>
> + %sv1 = call i32 @llvm.ctlz.i32(i32 %x)<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %dummy1 = add i32 %w, 6<br>
> + %sv2 = call i32 @llvm.cttz.i32(i32 %x)<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %p = phi i32 [ %sv1, %if.then ], [ %sv2, %if.else ]<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +declare i32 @llvm.ctlz.i32(i32 %x) readnone<br>
> +declare i32 @llvm.cttz.i32(i32 %x) readnone<br>
> +<br>
> +; CHECK-LABEL: test12<br>
> +; CHECK: call i32 @llvm.ctlz<br>
> +; CHECK: call i32 @llvm.cttz<br>
> +<br>
> +; The TBAA metadata should be properly combined.<br>
> +define i32 @test13(i1 zeroext %flag, i32 %x, i32* %y) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %z = load volatile i32, i32* %y<br>
> + %a = add i32 %z, 5<br>
> + store volatile i32 %a, i32* %y, !tbaa !3<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %w = load volatile i32, i32* %y<br>
> + %b = add i32 %w, 7<br>
> + store volatile i32 %b, i32* %y, !tbaa !4<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +!0 = !{ !"an example type tree" }<br>
> +!1 = !{ !"int", !0 }<br>
> +!2 = !{ !"float", !0 }<br>
> +!3 = !{ !"const float", !2, i64 0 }<br>
> +!4 = !{ !"special float", !2, i64 1 }<br>
> +<br>
> +; CHECK-LABEL: test13<br>
> +; CHECK-DAG: select<br>
> +; CHECK-DAG: load volatile<br>
> +; CHECK: store volatile {{.*}}, !tbaa !0<br>
> +; CHECK-NOT: load<br>
> +; CHECK-NOT: store<br>
> +<br>
> +; The call should be commoned.<br>
> +define i32 @test13a(i1 zeroext %flag, i32 %w, i32 %x, i32 %y) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %sv1 = call i32 @bar(i32 %x)<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %sv2 = call i32 @bar(i32 %y)<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %p = phi i32 [ %sv1, %if.then ], [ %sv2, %if.else ]<br>
> + ret i32 1<br>
> +}<br>
> +declare i32 @bar(i32)<br>
> +<br>
> +; CHECK-LABEL: test13a<br>
> +; CHECK: %[[x:.*]] = select i1 %flag<br>
> +; CHECK: call i32 @bar(i32 %[[x]])<br>
> +<br>
> +; The load should be commoned.<br>
> +define i32 @test14(i1 zeroext %flag, i32 %w, i32 %x, i32 %y, %struct.anon* %s) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %dummy = add i32 %x, 1<br>
> + %gepa = getelementptr inbounds %struct.anon, %struct.anon* %s, i32 0, i32 1<br>
> + %sv1 = load i32, i32* %gepa<br>
> + %cmp1 = icmp eq i32 %sv1, 56<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %dummy2 = add i32 %x, 4<br>
> + %gepb = getelementptr inbounds %struct.anon, %struct.anon* %s, i32 0, i32 1<br>
> + %sv2 = load i32, i32* %gepb<br>
> + %cmp2 = icmp eq i32 %sv2, 57<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %p = phi i1 [ %cmp1, %if.then ], [ %cmp2, %if.else ]<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test14<br>
> +; CHECK: getelementptr<br>
> +; CHECK: load<br>
> +; CHECK-NOT: load<br>
> +<br>
> +; The load should be commoned.<br>
> +define i32 @test15(i1 zeroext %flag, i32 %w, i32 %x, i32 %y, %struct.anon* %s) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %dummy = add i32 %x, 1<br>
> + %gepa = getelementptr inbounds %struct.anon, %struct.anon* %s, i32 0, i32 0<br>
> + %sv1 = load i32, i32* %gepa<br>
> + %ext1 = zext i32 %sv1 to i64<br>
> + %cmp1 = icmp eq i64 %ext1, 56<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %dummy2 = add i32 %x, 4<br>
> + %gepb = getelementptr inbounds %struct.anon, %struct.anon* %s, i32 0, i32 1<br>
> + %sv2 = load i32, i32* %gepb<br>
> + %ext2 = zext i32 %sv2 to i64<br>
> + %cmp2 = icmp eq i64 %ext2, 56<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %p = phi i1 [ %cmp1, %if.then ], [ %cmp2, %if.else ]<br>
> + ret i32 1<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test15<br>
> +; CHECK: getelementptr<br>
> +; CHECK: load<br>
> +; CHECK-NOT: load<br>
> +<br>
> +define zeroext i1 @test_crash(i1 zeroext %flag, i32* %i4, i32* %m, i32* %n) {<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %tmp1 = load i32, i32* %i4<br>
> + %tmp2 = add i32 %tmp1, -1<br>
> + store i32 %tmp2, i32* %i4<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + %tmp3 = load i32, i32* %m<br>
> + %tmp4 = load i32, i32* %n<br>
> + %tmp5 = add i32 %tmp3, %tmp4<br>
> + store i32 %tmp5, i32* %i4<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i1 true<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test_crash<br>
> +; No checks for test_crash - just ensure it doesn't crash!<br>
> +<br>
> +define zeroext i1 @test16(i1 zeroext %flag, i1 zeroext %flag2, i32 %blksA, i32 %blksB, i32 %nblks) {<br>
> +<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + %frombool1 = zext i1 %cmp to i8<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + br i1 %flag2, label %if.then2, label %if.end<br>
> +<br>
> +if.then2:<br>
> + %add = add i32 %nblks, %blksB<br>
> + %cmp2 = icmp ule i32 %add, %blksA<br>
> + %frombool3 = zext i1 %cmp2 to i8<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %obeys.0 = phi i8 [ %frombool1, %if.then ], [ %frombool3, %if.then2 ], [ 0, %if.else ]<br>
> + %tobool4 = icmp ne i8 %obeys.0, 0<br>
> + ret i1 %tobool4<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test16<br>
> +; CHECK: zext<br>
> +; CHECK: zext<br>
> +<br>
> +define zeroext i1 @test16a(i1 zeroext %flag, i1 zeroext %flag2, i32 %blksA, i32 %blksB, i32 %nblks, i8* %p) {<br>
> +<br>
> +entry:<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + %frombool1 = zext i1 %cmp to i8<br>
> + %b1 = sext i8 %frombool1 to i32<br>
> + %b2 = trunc i32 %b1 to i8<br>
> + store i8 %b2, i8* %p<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + br i1 %flag2, label %if.then2, label %if.end<br>
> +<br>
> +if.then2:<br>
> + %add = add i32 %nblks, %blksB<br>
> + %cmp2 = icmp ule i32 %add, %blksA<br>
> + %frombool3 = zext i1 %cmp2 to i8<br>
> + %a1 = sext i8 %frombool3 to i32<br>
> + %a2 = trunc i32 %a1 to i8<br>
> + store i8 %a2, i8* %p<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i1 true<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test16a<br>
> +; CHECK: zext<br>
> +; CHECK-NOT: zext<br>
> +<br>
> +define zeroext i1 @test17(i32 %flag, i32 %blksA, i32 %blksB, i32 %nblks) {<br>
> +entry:<br>
> + switch i32 %flag, label %if.end [<br>
> + i32 0, label %if.then<br>
> + i32 1, label %if.then2<br>
> + ]<br>
> +<br>
> +if.then:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + %frombool1 = call i8 @i1toi8(i1 %cmp)<br>
> + %a1 = sext i8 %frombool1 to i32<br>
> + %a2 = trunc i32 %a1 to i8<br>
> + br label %if.end<br>
> +<br>
> +if.then2:<br>
> + %add = add i32 %nblks, %blksB<br>
> + %cmp2 = icmp ule i32 %add, %blksA<br>
> + %frombool3 = call i8 @i1toi8(i1 %cmp2)<br>
> + %b1 = sext i8 %frombool3 to i32<br>
> + %b2 = trunc i32 %b1 to i8<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %obeys.0 = phi i8 [ %a2, %if.then ], [ %b2, %if.then2 ], [ 0, %entry ]<br>
> + %tobool4 = icmp ne i8 %obeys.0, 0<br>
> + ret i1 %tobool4<br>
> +}<br>
> +declare i8 @i1toi8(i1)<br>
> +<br>
> +; FIXME: DISABLED - we don't consider this profitable. We should<br>
> +; - Consider argument setup/return mov'ing for calls, like InlineCost does.<br>
> +; - Consider the removal of the %obeys.0 PHI (zero PHI movement overall)<br>
> +<br>
> +; DISABLED-CHECK-LABEL: test17<br>
> +; DISABLED-CHECK: if.then:<br>
> +; DISABLED-CHECK-NEXT: icmp uge<br>
> +; DISABLED-CHECK-NEXT: br label %[[x:.*]]<br>
> +<br>
> +; DISABLED-CHECK: if.then2:<br>
> +; DISABLED-CHECK-NEXT: add<br>
> +; DISABLED-CHECK-NEXT: icmp ule<br>
> +; DISABLED-CHECK-NEXT: br label %[[x]]<br>
> +<br>
> +; DISABLED-CHECK: [[x]]:<br>
> +; DISABLED-CHECK-NEXT: %[[y:.*]] = phi i1 [ %cmp<br>
> +; DISABLED-CHECK-NEXT: %[[z:.*]] = call i8 @i1toi8(i1 %[[y]])<br>
> +; DISABLED-CHECK-NEXT: br label %if.end<br>
> +<br>
> +; DISABLED-CHECK: if.end:<br>
> +; DISABLED-CHECK-NEXT: phi i8<br>
> +; DISABLED-CHECK-DAG: [ %[[z]], %[[x]] ]<br>
> +; DISABLED-CHECK-DAG: [ 0, %entry ]<br>
> +<br>
> +define zeroext i1 @test18(i32 %flag, i32 %blksA, i32 %blksB, i32 %nblks) {<br>
> +entry:<br>
> + switch i32 %flag, label %if.then3 [<br>
> + i32 0, label %if.then<br>
> + i32 1, label %if.then2<br>
> + ]<br>
> +<br>
> +if.then:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + %frombool1 = zext i1 %cmp to i8<br>
> + br label %if.end<br>
> +<br>
> +if.then2:<br>
> + %add = add i32 %nblks, %blksB<br>
> + %cmp2 = icmp ule i32 %add, %blksA<br>
> + %frombool3 = zext i1 %cmp2 to i8<br>
> + br label %if.end<br>
> +<br>
> +if.then3:<br>
> + %add2 = add i32 %nblks, %blksA<br>
> + %cmp3 = icmp ule i32 %add2, %blksA<br>
> + %frombool4 = zext i1 %cmp3 to i8<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + %obeys.0 = phi i8 [ %frombool1, %if.then ], [ %frombool3, %if.then2 ], [ %frombool4, %if.then3 ]<br>
> + %tobool4 = icmp ne i8 %obeys.0, 0<br>
> + ret i1 %tobool4<br>
> +}<br>
> +<br>
> +; CHECK-LABEL: test18<br>
> +; CHECK: if.end:<br>
> +; CHECK-NEXT: %[[x:.*]] = phi i1<br>
> +; CHECK-DAG: [ %cmp, %if.then ]<br>
> +; CHECK-DAG: [ %cmp2, %if.then2 ]<br>
> +; CHECK-DAG: [ %cmp3, %if.then3 ]<br>
> +; CHECK-NEXT: zext i1 %[[x]] to i8<br>
> +<br>
> +; The phi is confusing - both add instructions are used by it, but<br>
> +; not on their respective unconditional arcs. It should not be<br>
> +; optimized.<br>
> +define void @test_pr30292(i1 %cond, i1 %cond2, i32 %a, i32 %b) {<br>
> +entry:<br>
> + %add1 = add i32 %a, 1<br>
> + br label %succ<br>
> +<br>
> +one:<br>
> + br i1 %cond, label %two, label %succ<br>
> +<br>
> +two:<br>
> + call void @g()<br>
> + %add2 = add i32 %a, 1<br>
> + br label %succ<br>
> +<br>
> +succ:<br>
> + %p = phi i32 [ 0, %entry ], [ %add1, %one ], [ %add2, %two ]<br>
> + br label %one<br>
> +}<br>
> +declare void @g()<br>
> +<br>
> +; CHECK-LABEL: test_pr30292<br>
> +; CHECK: phi i32 [ 0, %entry ], [ %add1, %succ ], [ %add2, %two ]<br>
> +<br>
> +define zeroext i1 @test_pr30244(i1 zeroext %flag, i1 zeroext %flag2, i32 %blksA, i32 %blksB, i32 %nblks) {<br>
> +<br>
> +entry:<br>
> + %p = alloca i8<br>
> + br i1 %flag, label %if.then, label %if.else<br>
> +<br>
> +if.then:<br>
> + %cmp = icmp uge i32 %blksA, %nblks<br>
> + %frombool1 = zext i1 %cmp to i8<br>
> + store i8 %frombool1, i8* %p<br>
> + br label %if.end<br>
> +<br>
> +if.else:<br>
> + br i1 %flag2, label %if.then2, label %if.end<br>
> +<br>
> +if.then2:<br>
> + %add = add i32 %nblks, %blksB<br>
> + %cmp2 = icmp ule i32 %add, %blksA<br>
> + %frombool3 = zext i1 %cmp2 to i8<br>
> + store i8 %frombool3, i8* %p<br>
> + br label %if.end<br>
> +<br>
> +if.end:<br>
> + ret i1 tr</blockquote></div></div></div></div>