<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle19
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">My testing with MSVC could take a while and I don’t want to block your progress so I committed r332910 with the fix after testing it
with gcc and clang.<o:p></o:p></span></a></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Please, let me know if it’s not working for you.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Diego<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> llvm-commits [mailto:llvm-commits-bounces@lists.llvm.org]
<b>On Behalf Of </b>Caballero, Diego via llvm-commits<br>
<b>Sent:</b> Monday, May 21, 2018 1:45 PM<br>
<b>To:</b> Greg Bedwell <gregbedwell@gmail.com><br>
<b>Cc:</b> llvm-commits <llvm-commits@lists.llvm.org><br>
<b>Subject:</b> RE: [llvm] r332860 - [VPlan] Reland r332654 and silence unused func warning<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Ok, I’m having a look.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Diego<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Greg Bedwell [<a href="mailto:gregbedwell@gmail.com">mailto:gregbedwell@gmail.com</a>]
<br>
<b>Sent:</b> Monday, May 21, 2018 1:18 PM<br>
<b>To:</b> Caballero, Diego <<a href="mailto:diego.caballero@intel.com">diego.caballero@intel.com</a>><br>
<b>Cc:</b> llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm] r332860 - [VPlan] Reland r332654 and silence unused func warning<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">It looks like MSVC still doesn't like this unfortunately.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'm building RelWithDebInfo with "19.00.24213.1 for x64" which reports the following for the line
<span style="font-family:"Arial",sans-serif;color:#222222;background:white">"(void)hasDuplicates;"</span>:<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">E:\work\upstream-llvm\llvm\lib\Transforms\Vectorize\VPlanVerifier.cpp(55): warning C4551: function call missing argument list<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On 21 May 2018 at 19:14, Diego Caballero via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal">Author: dcaballe<br>
Date: Mon May 21 11:14:23 2018<br>
New Revision: 332860<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=332860&view=rev" target="_blank">
http://llvm.org/viewvc/llvm-project?rev=332860&view=rev</a><br>
Log:<br>
[VPlan] Reland r332654 and silence unused func warning<br>
<br>
r332654 was reverted due to an unused function warning in<br>
release build. This commit includes the same code with the<br>
warning silenced.<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D44338" target="_blank">
https://reviews.llvm.org/D44338</a><br>
<br>
<br>
Added:<br>
llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp (with props)<br>
llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.h (with props)<br>
llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.cpp (with props)<br>
llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.h (with props)<br>
llvm/trunk/test/Transforms/LoopVectorize/vplan_hcfg_stress_test.ll<br>
Modified:<br>
llvm/trunk/lib/Transforms/Vectorize/CMakeLists.txt<br>
llvm/trunk/lib/Transforms/Vectorize/LoopVectorizationPlanner.h<br>
llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
llvm/trunk/lib/Transforms/Vectorize/VPlan.h<br>
llvm/trunk/lib/Transforms/Vectorize/VPlanValue.h<br>
<br>
Modified: llvm/trunk/lib/Transforms/Vectorize/CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/CMakeLists.txt?rev=332860&r1=332859&r2=332860&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/CMakeLists.txt?rev=332860&r1=332859&r2=332860&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/CMakeLists.txt (original)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/CMakeLists.txt Mon May 21 11:14:23 2018<br>
@@ -5,6 +5,8 @@ add_llvm_library(LLVMVectorize<br>
SLPVectorizer.cpp<br>
Vectorize.cpp<br>
VPlan.cpp<br>
+ VPlanHCFGBuilder.cpp<br>
+ VPlanVerifier.cpp<br>
<br>
ADDITIONAL_HEADER_DIRS<br>
${LLVM_MAIN_INCLUDE_DIR}/llvm/Transforms<br>
<br>
Modified: llvm/trunk/lib/Transforms/Vectorize/LoopVectorizationPlanner.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/LoopVectorizationPlanner.h?rev=332860&r1=332859&r2=332860&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/LoopVectorizationPlanner.h?rev=332860&r1=332859&r2=332860&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/LoopVectorizationPlanner.h (original)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/LoopVectorizationPlanner.h Mon May 21 11:14:23 2018<br>
@@ -39,23 +39,94 @@ private:<br>
VPBasicBlock::iterator InsertPt = VPBasicBlock::iterator();<br>
<br>
VPInstruction *createInstruction(unsigned Opcode,<br>
- std::initializer_list<VPValue *> Operands) {<br>
+ ArrayRef<VPValue *> Operands) {<br>
VPInstruction *Instr = new VPInstruction(Opcode, Operands);<br>
- BB->insert(Instr, InsertPt);<br>
+ if (BB)<br>
+ BB->insert(Instr, InsertPt);<br>
return Instr;<br>
}<br>
<br>
+ VPInstruction *createInstruction(unsigned Opcode,<br>
+ std::initializer_list<VPValue *> Operands) {<br>
+ return createInstruction(Opcode, ArrayRef<VPValue *>(Operands));<br>
+ }<br>
+<br>
public:<br>
VPBuilder() {}<br>
<br>
- /// This specifies that created VPInstructions should be appended to<br>
- /// the end of the specified block.<br>
+ /// Clear the insertion point: created instructions will not be inserted into<br>
+ /// a block.<br>
+ void clearInsertionPoint() {<br>
+ BB = nullptr;<br>
+ InsertPt = VPBasicBlock::iterator();<br>
+ }<br>
+<br>
+ VPBasicBlock *getInsertBlock() const { return BB; }<br>
+ VPBasicBlock::iterator getInsertPoint() const { return InsertPt; }<br>
+<br>
+ /// InsertPoint - A saved insertion point.<br>
+ class VPInsertPoint {<br>
+ VPBasicBlock *Block = nullptr;<br>
+ VPBasicBlock::iterator Point;<br>
+<br>
+ public:<br>
+ /// Creates a new insertion point which doesn't point to anything.<br>
+ VPInsertPoint() = default;<br>
+<br>
+ /// Creates a new insertion point at the given location.<br>
+ VPInsertPoint(VPBasicBlock *InsertBlock, VPBasicBlock::iterator InsertPoint)<br>
+ : Block(InsertBlock), Point(InsertPoint) {}<br>
+<br>
+ /// Returns true if this insert point is set.<br>
+ bool isSet() const { return Block != nullptr; }<br>
+<br>
+ VPBasicBlock *getBlock() const { return Block; }<br>
+ VPBasicBlock::iterator getPoint() const { return Point; }<br>
+ };<br>
+<br>
+ /// Sets the current insert point to a previously-saved location.<br>
+ void restoreIP(VPInsertPoint IP) {<br>
+ if (IP.isSet())<br>
+ setInsertPoint(IP.getBlock(), IP.getPoint());<br>
+ else<br>
+ clearInsertionPoint();<br>
+ }<br>
+<br>
+ /// This specifies that created VPInstructions should be appended to the end<br>
+ /// of the specified block.<br>
void setInsertPoint(VPBasicBlock *TheBB) {<br>
assert(TheBB && "Attempting to set a null insert point");<br>
BB = TheBB;<br>
InsertPt = BB->end();<br>
}<br>
<br>
+ /// This specifies that created instructions should be inserted at the<br>
+ /// specified point.<br>
+ void setInsertPoint(VPBasicBlock *TheBB, VPBasicBlock::iterator IP) {<br>
+ BB = TheBB;<br>
+ InsertPt = IP;<br>
+ }<br>
+<br>
+ /// Insert and return the specified instruction.<br>
+ VPInstruction *insert(VPInstruction *I) const {<br>
+ BB->insert(I, InsertPt);<br>
+ return I;<br>
+ }<br>
+<br>
+ /// Create an N-ary operation with \p Opcode, \p Operands and set \p Inst as<br>
+ /// its underlying Instruction.<br>
+ VPValue *createNaryOp(unsigned Opcode, ArrayRef<VPValue *> Operands,<br>
+ Instruction *Inst = nullptr) {<br>
+ VPInstruction *NewVPInst = createInstruction(Opcode, Operands);<br>
+ NewVPInst->setUnderlyingValue(Inst);<br>
+ return NewVPInst;<br>
+ }<br>
+ VPValue *createNaryOp(unsigned Opcode,<br>
+ std::initializer_list<VPValue *> Operands,<br>
+ Instruction *Inst = nullptr) {<br>
+ return createNaryOp(Opcode, ArrayRef<VPValue *>(Operands), Inst);<br>
+ }<br>
+<br>
VPValue *createNot(VPValue *Operand) {<br>
return createInstruction(VPInstruction::Not, {Operand});<br>
}<br>
@@ -67,8 +138,28 @@ public:<br>
VPValue *createOr(VPValue *LHS, VPValue *RHS) {<br>
return createInstruction(Instruction::BinaryOps::Or, {LHS, RHS});<br>
}<br>
-};<br>
<br>
+ //===--------------------------------------------------------------------===//<br>
+ // RAII helpers.<br>
+ //===--------------------------------------------------------------------===//<br>
+<br>
+ /// RAII object that stores the current insertion point and restores it when<br>
+ /// the object is destroyed.<br>
+ class InsertPointGuard {<br>
+ VPBuilder &Builder;<br>
+ VPBasicBlock *Block;<br>
+ VPBasicBlock::iterator Point;<br>
+<br>
+ public:<br>
+ InsertPointGuard(VPBuilder &B)<br>
+ : Builder(B), Block(B.getInsertBlock()), Point(B.getInsertPoint()) {}<br>
+<br>
+ InsertPointGuard(const InsertPointGuard &) = delete;<br>
+ InsertPointGuard &operator=(const InsertPointGuard &) = delete;<br>
+<br>
+ ~InsertPointGuard() { Builder.restoreIP(VPInsertPoint(Block, Point)); }<br>
+ };<br>
+};<br>
<br>
/// TODO: The following VectorizationFactor was pulled out of<br>
/// LoopVectorizationCostModel class. LV also deals with<br>
<br>
Modified: llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp?rev=332860&r1=332859&r2=332860&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp?rev=332860&r1=332859&r2=332860&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp (original)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp Mon May 21 11:14:23 2018<br>
@@ -56,6 +56,7 @@<br>
<br>
#include "llvm/Transforms/Vectorize/LoopVectorize.h"<br>
#include "LoopVectorizationPlanner.h"<br>
+#include "VPlanHCFGBuilder.h"<br>
#include "llvm/ADT/APInt.h"<br>
#include "llvm/ADT/ArrayRef.h"<br>
#include "llvm/ADT/DenseMap.h"<br>
@@ -244,6 +245,17 @@ static cl::opt<bool> EnableVPlanNativePa<br>
cl::desc("Enable VPlan-native vectorization path with "<br>
"support for outer loop vectorization."));<br>
<br>
+// This flag enables the stress testing of the VPlan H-CFG construction in the<br>
+// VPlan-native vectorization path. It must be used in conjuction with<br>
+// -enable-vplan-native-path. -vplan-verify-hcfg can also be used to enable the<br>
+// verification of the H-CFGs built.<br>
+static cl::opt<bool> VPlanBuildStressTest(<br>
+ "vplan-build-stress-test", cl::init(false), cl::Hidden,<br>
+ cl::desc(<br>
+ "Build VPlan for every supported loop nest in the function and bail "<br>
+ "out right after the build (stress test the VPlan H-CFG construction "<br>
+ "in the VPlan-native vectorization path)."));<br>
+<br>
/// A helper function for converting Scalar types to vector types.<br>
/// If the incoming type is void, we return void. If the VF is 1, we return<br>
/// the scalar type.<br>
@@ -1653,8 +1665,11 @@ static void collectSupportedLoops(Loop &<br>
OptimizationRemarkEmitter *ORE,<br>
SmallVectorImpl<Loop *> &V) {<br>
// Collect inner loops and outer loops without irreducible control flow. For<br>
- // now, only collect outer loops that have explicit vectorization hints.<br>
- if (L.empty() || (EnableVPlanNativePath && isExplicitVecOuterLoop(&L, ORE))) {<br>
+ // now, only collect outer loops that have explicit vectorization hints. If we<br>
+ // are stress testing the VPlan H-CFG construction, we collect the outermost<br>
+ // loop of every loop nest.<br>
+ if (L.empty() || VPlanBuildStressTest ||<br>
+ (EnableVPlanNativePath && isExplicitVecOuterLoop(&L, ORE))) {<br>
LoopBlocksRPO RPOT(&L);<br>
RPOT.perform(LI);<br>
if (!containsIrreducibleCFG<const BasicBlock *>(RPOT, *LI)) {<br>
@@ -6254,7 +6269,7 @@ void LoopVectorizationCostModel::collect<br>
VectorizationFactor<br>
LoopVectorizationPlanner::planInVPlanNativePath(bool OptForSize,<br>
unsigned UserVF) {<br>
- // Width 1 means no vectorize, cost 0 means uncomputed cost.<br>
+ // Width 1 means no vectorization, cost 0 means uncomputed cost.<br>
const VectorizationFactor NoVectorization = {1U, 0U};<br>
<br>
// Outer loop handling: They may require CFG and instruction level<br>
@@ -6262,12 +6277,22 @@ LoopVectorizationPlanner::planInVPlanNat<br>
// Since we cannot modify the incoming IR, we need to build VPlan upfront in<br>
// the vectorization pipeline.<br>
if (!OrigLoop->empty()) {<br>
+ // TODO: If UserVF is not provided, we set UserVF to 4 for stress testing.<br>
+ // This won't be necessary when UserVF is not required in the VPlan-native<br>
+ // path.<br>
+ if (VPlanBuildStressTest && !UserVF)<br>
+ UserVF = 4;<br>
+<br>
assert(EnableVPlanNativePath && "VPlan-native path is not enabled.");<br>
assert(UserVF && "Expected UserVF for outer loop vectorization.");<br>
assert(isPowerOf2_32(UserVF) && "VF needs to be a power of two");<br>
LLVM_DEBUG(dbgs() << "LV: Using user VF " << UserVF << ".\n");<br>
buildVPlans(UserVF, UserVF);<br>
<br>
+ // For VPlan build stress testing, we bail out after VPlan construction.<br>
+ if (VPlanBuildStressTest)<br>
+ return NoVectorization;<br>
+<br>
return {UserVF, 0};<br>
}<br>
<br>
@@ -6280,7 +6305,7 @@ LoopVectorizationPlanner::planInVPlanNat<br>
VectorizationFactor<br>
LoopVectorizationPlanner::plan(bool OptForSize, unsigned UserVF) {<br>
assert(OrigLoop->empty() && "Inner loop expected.");<br>
- // Width 1 means no vectorize, cost 0 means uncomputed cost.<br>
+ // Width 1 means no vectorization, cost 0 means uncomputed cost.<br>
const VectorizationFactor NoVectorization = {1U, 0U};<br>
Optional<unsigned> MaybeMaxVF = CM.computeMaxVF(OptForSize);<br>
if (!MaybeMaxVF.hasValue()) // Cases considered too costly to vectorize.<br>
@@ -6806,9 +6831,11 @@ VPBasicBlock *LoopVectorizationPlanner::<br>
"VPBB has successors when handling predicated replication.");<br>
// Record predicated instructions for above packing optimizations.<br>
PredInst2Recipe[I] = Recipe;<br>
- VPBlockBase *Region =<br>
- VPBB->setOneSuccessor(createReplicateRegion(I, Recipe, Plan));<br>
- return cast<VPBasicBlock>(Region->setOneSuccessor(new VPBasicBlock()));<br>
+ VPBlockBase *Region = createReplicateRegion(I, Recipe, Plan);<br>
+ VPBlockUtils::insertBlockAfter(Region, VPBB);<br>
+ auto *RegSucc = new VPBasicBlock();<br>
+ VPBlockUtils::insertBlockAfter(RegSucc, Region);<br>
+ return RegSucc;<br>
}<br>
<br>
VPRegionBlock *<br>
@@ -6834,8 +6861,8 @@ LoopVectorizationPlanner::createReplicat<br>
<br>
// Note: first set Entry as region entry and then connect successors starting<br>
// from it in order, to propagate the "parent" of each VPBasicBlock.<br>
- Entry->setTwoSuccessors(Pred, Exit);<br>
- Pred->setOneSuccessor(Exit);<br>
+ VPBlockUtils::insertTwoBlocksAfter(Pred, Exit, Entry);<br>
+ VPBlockUtils::connectBlocks(Pred, Exit);<br>
<br>
return Region;<br>
}<br>
@@ -6852,6 +6879,11 @@ LoopVectorizationPlanner::buildVPlan(VFR<br>
<br>
// Create new empty VPlan<br>
auto Plan = llvm::make_unique<VPlan>();<br>
+<br>
+ // Build hierarchical CFG<br>
+ VPlanHCFGBuilder HCFGBuilder(OrigLoop, LI);<br>
+ HCFGBuilder.buildHierarchicalCFG(*Plan.get());<br>
+<br>
return Plan;<br>
}<br>
<br>
@@ -6893,7 +6925,7 @@ LoopVectorizationPlanner::buildVPlan(VFR<br>
// ingredients and fill a new VPBasicBlock.<br>
unsigned VPBBsForBB = 0;<br>
auto *FirstVPBBForBB = new VPBasicBlock(BB->getName());<br>
- VPBB->setOneSuccessor(FirstVPBBForBB);<br>
+ VPBlockUtils::insertBlockAfter(FirstVPBBForBB, VPBB);<br>
VPBB = FirstVPBBForBB;<br>
Builder.setInsertPoint(VPBB);<br>
<br>
@@ -6997,7 +7029,7 @@ LoopVectorizationPlanner::buildVPlan(VFR<br>
VPBasicBlock *PreEntry = cast<VPBasicBlock>(Plan->getEntry());<br>
assert(PreEntry->empty() && "Expecting empty pre-entry block.");<br>
VPBlockBase *Entry = Plan->setEntry(PreEntry->getSingleSuccessor());<br>
- PreEntry->disconnectSuccessor(Entry);<br>
+ VPBlockUtils::disconnectBlocks(PreEntry, Entry);<br>
delete PreEntry;<br>
<br>
std::string PlanName;<br>
<br>
Modified: llvm/trunk/lib/Transforms/Vectorize/VPlan.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlan.h?rev=332860&r1=332859&r2=332860&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlan.h?rev=332860&r1=332859&r2=332860&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/VPlan.h (original)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/VPlan.h Mon May 21 11:14:23 2018<br>
@@ -306,6 +306,8 @@ struct VPTransformState {<br>
/// VPBlockBase is the building block of the Hierarchical Control-Flow Graph.<br>
/// A VPBlockBase can be either a VPBasicBlock or a VPRegionBlock.<br>
class VPBlockBase {<br>
+ friend class VPBlockUtils;<br>
+<br>
private:<br>
const unsigned char SubclassID; ///< Subclass identifier (for isa/dyn_cast).<br>
<br>
@@ -372,6 +374,7 @@ public:<br>
/// for any other purpose, as the values may change as LLVM evolves.<br>
unsigned getVPBlockID() const { return SubclassID; }<br>
<br>
+ VPRegionBlock *getParent() { return Parent; }<br>
const VPRegionBlock *getParent() const { return Parent; }<br>
<br>
void setParent(VPRegionBlock *P) { Parent = P; }<br>
@@ -406,6 +409,9 @@ public:<br>
return (Predecessors.size() == 1 ? *Predecessors.begin() : nullptr);<br>
}<br>
<br>
+ size_t getNumSuccessors() const { return Successors.size(); }<br>
+ size_t getNumPredecessors() const { return Predecessors.size(); }<br>
+<br>
/// An Enclosing Block of a block B is any block containing B, including B<br>
/// itself. \return the closest enclosing block starting from "this", which<br>
/// has successors. \return the root enclosing block if all enclosing blocks<br>
@@ -449,34 +455,31 @@ public:<br>
return getEnclosingBlockWithPredecessors()->getSinglePredecessor();<br>
}<br>
<br>
- /// Sets a given VPBlockBase \p Successor as the single successor and \return<br>
- /// \p Successor. The parent of this Block is copied to be the parent of<br>
- /// \p Successor.<br>
- VPBlockBase *setOneSuccessor(VPBlockBase *Successor) {<br>
+ /// Set a given VPBlockBase \p Successor as the single successor of this<br>
+ /// VPBlockBase. This VPBlockBase is not added as predecessor of \p Successor.<br>
+ /// This VPBlockBase must have no successors.<br>
+ void setOneSuccessor(VPBlockBase *Successor) {<br>
assert(Successors.empty() && "Setting one successor when others exist.");<br>
appendSuccessor(Successor);<br>
- Successor->appendPredecessor(this);<br>
- Successor->Parent = Parent;<br>
- return Successor;<br>
}<br>
<br>
- /// Sets two given VPBlockBases \p IfTrue and \p IfFalse to be the two<br>
- /// successors. The parent of this Block is copied to be the parent of both<br>
- /// \p IfTrue and \p IfFalse.<br>
+ /// Set two given VPBlockBases \p IfTrue and \p IfFalse to be the two<br>
+ /// successors of this VPBlockBase. This VPBlockBase is not added as<br>
+ /// predecessor of \p IfTrue or \p IfFalse. This VPBlockBase must have no<br>
+ /// successors.<br>
void setTwoSuccessors(VPBlockBase *IfTrue, VPBlockBase *IfFalse) {<br>
assert(Successors.empty() && "Setting two successors when others exist.");<br>
appendSuccessor(IfTrue);<br>
appendSuccessor(IfFalse);<br>
- IfTrue->appendPredecessor(this);<br>
- IfFalse->appendPredecessor(this);<br>
- IfTrue->Parent = Parent;<br>
- IfFalse->Parent = Parent;<br>
}<br>
<br>
- void disconnectSuccessor(VPBlockBase *Successor) {<br>
- assert(Successor && "Successor to disconnect is null.");<br>
- removeSuccessor(Successor);<br>
- Successor->removePredecessor(this);<br>
+ /// Set each VPBasicBlock in \p NewPreds as predecessor of this VPBlockBase.<br>
+ /// This VPBlockBase must have no predecessors. This VPBlockBase is not added<br>
+ /// as successor of any VPBasicBlock in \p NewPreds.<br>
+ void setPredecessors(ArrayRef<VPBlockBase *> NewPreds) {<br>
+ assert(Predecessors.empty() && "Block predecessors already set.");<br>
+ for (auto *Pred : NewPreds)<br>
+ appendPredecessor(Pred);<br>
}<br>
<br>
/// The method which generates the output IR that correspond to this<br>
@@ -554,10 +557,13 @@ private:<br>
void generateInstruction(VPTransformState &State, unsigned Part);<br>
<br>
public:<br>
- VPInstruction(unsigned Opcode, std::initializer_list<VPValue *> Operands)<br>
+ VPInstruction(unsigned Opcode, ArrayRef<VPValue *> Operands)<br>
: VPUser(VPValue::VPInstructionSC, Operands),<br>
VPRecipeBase(VPRecipeBase::VPInstructionSC), Opcode(Opcode) {}<br>
<br>
+ VPInstruction(unsigned Opcode, std::initializer_list<VPValue *> Operands)<br>
+ : VPInstruction(Opcode, ArrayRef<VPValue *>(Operands)) {}<br>
+<br>
/// Method to support type inquiry through isa, cast, and dyn_cast.<br>
static inline bool classof(const VPValue *V) {<br>
return V->getVPValueID() == VPValue::VPInstructionSC;<br>
@@ -963,6 +969,9 @@ public:<br>
Entry->setParent(this);<br>
Exit->setParent(this);<br>
}<br>
+ VPRegionBlock(const std::string &Name = "", bool IsReplicator = false)<br>
+ : VPBlockBase(VPRegionBlockSC, Name), Entry(nullptr), Exit(nullptr),<br>
+ IsReplicator(IsReplicator) {}<br>
<br>
~VPRegionBlock() override {<br>
if (Entry)<br>
@@ -977,9 +986,27 @@ public:<br>
const VPBlockBase *getEntry() const { return Entry; }<br>
VPBlockBase *getEntry() { return Entry; }<br>
<br>
+ /// Set \p EntryBlock as the entry VPBlockBase of this VPRegionBlock. \p<br>
+ /// EntryBlock must have no predecessors.<br>
+ void setEntry(VPBlockBase *EntryBlock) {<br>
+ assert(EntryBlock->getPredecessors().empty() &&<br>
+ "Entry block cannot have predecessors.");<br>
+ Entry = EntryBlock;<br>
+ EntryBlock->setParent(this);<br>
+ }<br>
+<br>
const VPBlockBase *getExit() const { return Exit; }<br>
VPBlockBase *getExit() { return Exit; }<br>
<br>
+ /// Set \p ExitBlock as the exit VPBlockBase of this VPRegionBlock. \p<br>
+ /// ExitBlock must have no successors.<br>
+ void setExit(VPBlockBase *ExitBlock) {<br>
+ assert(ExitBlock->getSuccessors().empty() &&<br>
+ "Exit block cannot have successors.");<br>
+ Exit = ExitBlock;<br>
+ ExitBlock->setParent(this);<br>
+ }<br>
+<br>
/// An indicator whether this region is to generate multiple replicated<br>
/// instances of output IR corresponding to its VPBlockBases.<br>
bool isReplicator() const { return IsReplicator; }<br>
@@ -1007,6 +1034,13 @@ private:<br>
/// Holds the name of the VPlan, for printing.<br>
std::string Name;<br>
<br>
+ /// Holds all the external definitions created for this VPlan.<br>
+ // TODO: Introduce a specific representation for external definitions in<br>
+ // VPlan. External definitions must be immutable and hold a pointer to its<br>
+ // underlying IR that will be used to implement its structural comparison<br>
+ // (operators '==' and '<').<br>
+ SmallSet<VPValue *, 16> VPExternalDefs;<br>
+<br>
/// Holds a mapping between Values and their corresponding VPValue inside<br>
/// VPlan.<br>
Value2VPValueTy Value2VPValue;<br>
@@ -1019,6 +1053,8 @@ public:<br>
VPBlockBase::deleteCFG(Entry);<br>
for (auto &MapEntry : Value2VPValue)<br>
delete MapEntry.second;<br>
+ for (VPValue *Def : VPExternalDefs)<br>
+ delete Def;<br>
}<br>
<br>
/// Generate the IR code for this VPlan.<br>
@@ -1037,6 +1073,12 @@ public:<br>
<br>
void setName(const Twine &newName) { Name = newName.str(); }<br>
<br>
+ /// Add \p VPVal to the pool of external definitions if it's not already<br>
+ /// in the pool.<br>
+ void addExternalDef(VPValue *VPVal) {<br>
+ VPExternalDefs.insert(VPVal);<br>
+ }<br>
+<br>
void addVPValue(Value *V) {<br>
assert(V && "Trying to add a null Value to VPlan");<br>
assert(!Value2VPValue.count(V) && "Value already exists in VPlan");<br>
@@ -1184,6 +1226,70 @@ template <> struct GraphTraits<Inverse<V<br>
}<br>
};<br>
<br>
+//===----------------------------------------------------------------------===//<br>
+// VPlan Utilities<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+/// Class that provides utilities for VPBlockBases in VPlan.<br>
+class VPBlockUtils {<br>
+public:<br>
+ VPBlockUtils() = delete;<br>
+<br>
+ /// Insert disconnected VPBlockBase \p NewBlock after \p BlockPtr. Add \p<br>
+ /// NewBlock as successor of \p BlockPtr and \p Block as predecessor of \p<br>
+ /// NewBlock, and propagate \p BlockPtr parent to \p NewBlock. \p NewBlock<br>
+ /// must have neither successors nor predecessors.<br>
+ static void insertBlockAfter(VPBlockBase *NewBlock, VPBlockBase *BlockPtr) {<br>
+ assert(NewBlock->getSuccessors().empty() &&<br>
+ "Can't insert new block with successors.");<br>
+ // TODO: move successors from BlockPtr to NewBlock when this functionality<br>
+ // is necessary. For now, setBlockSingleSuccessor will assert if BlockPtr<br>
+ // already has successors.<br>
+ BlockPtr->setOneSuccessor(NewBlock);<br>
+ NewBlock->setPredecessors({BlockPtr});<br>
+ NewBlock->setParent(BlockPtr->getParent());<br>
+ }<br>
+<br>
+ /// Insert disconnected VPBlockBases \p IfTrue and \p IfFalse after \p<br>
+ /// BlockPtr. Add \p IfTrue and \p IfFalse as succesors of \p BlockPtr and \p<br>
+ /// BlockPtr as predecessor of \p IfTrue and \p IfFalse. Propagate \p BlockPtr<br>
+ /// parent to \p IfTrue and \p IfFalse. \p BlockPtr must have no successors<br>
+ /// and \p IfTrue and \p IfFalse must have neither successors nor<br>
+ /// predecessors.<br>
+ static void insertTwoBlocksAfter(VPBlockBase *IfTrue, VPBlockBase *IfFalse,<br>
+ VPBlockBase *BlockPtr) {<br>
+ assert(IfTrue->getSuccessors().empty() &&<br>
+ "Can't insert IfTrue with successors.");<br>
+ assert(IfFalse->getSuccessors().empty() &&<br>
+ "Can't insert IfFalse with successors.");<br>
+ BlockPtr->setTwoSuccessors(IfTrue, IfFalse);<br>
+ IfTrue->setPredecessors({BlockPtr});<br>
+ IfFalse->setPredecessors({BlockPtr});<br>
+ IfTrue->setParent(BlockPtr->getParent());<br>
+ IfFalse->setParent(BlockPtr->getParent());<br>
+ }<br>
+<br>
+ /// Connect VPBlockBases \p From and \p To bi-directionally. Append \p To to<br>
+ /// the successors of \p From and \p From to the predecessors of \p To. Both<br>
+ /// VPBlockBases must have the same parent, which can be null. Both<br>
+ /// VPBlockBases can be already connected to other VPBlockBases.<br>
+ static void connectBlocks(VPBlockBase *From, VPBlockBase *To) {<br>
+ assert((From->getParent() == To->getParent()) &&<br>
+ "Can't connect two block with different parents");<br>
+ assert(From->getNumSuccessors() < 2 &&<br>
+ "Blocks can't have more than two successors.");<br>
+ From->appendSuccessor(To);<br>
+ To->appendPredecessor(From);<br>
+ }<br>
+<br>
+ /// Disconnect VPBlockBases \p From and \p To bi-directionally. Remove \p To<br>
+ /// from the successors of \p From and \p From from the predecessors of \p To.<br>
+ static void disconnectBlocks(VPBlockBase *From, VPBlockBase *To) {<br>
+ assert(To && "Successor to disconnect is null.");<br>
+ From->removeSuccessor(To);<br>
+ To->removePredecessor(From);<br>
+ }<br>
+};<br>
} // end namespace llvm<br>
<br>
#endif // LLVM_TRANSFORMS_VECTORIZE_VPLAN_H<br>
<br>
Added: llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp?rev=332860&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp?rev=332860&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp (added)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp Mon May 21 11:14:23 2018<br>
@@ -0,0 +1,320 @@<br>
+//===-- VPlanHCFGBuilder.cpp ----------------------------------------------===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+///<br>
+/// \file<br>
+/// This file implements the construction of a VPlan-based Hierarchical CFG<br>
+/// (H-CFG) for an incoming IR. This construction comprises the following<br>
+/// components and steps:<br>
+//<br>
+/// 1. PlainCFGBuilder class: builds a plain VPBasicBlock-based CFG that<br>
+/// faithfully represents the CFG in the incoming IR. A VPRegionBlock (Top<br>
+/// Region) is created to enclose and serve as parent of all the VPBasicBlocks<br>
+/// in the plain CFG.<br>
+/// NOTE: At this point, there is a direct correspondence between all the<br>
+/// VPBasicBlocks created for the initial plain CFG and the incoming<br>
+/// BasicBlocks. However, this might change in the future.<br>
+///<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include "VPlanHCFGBuilder.h"<br>
+#include "LoopVectorizationPlanner.h"<br>
+#include "llvm/Analysis/LoopIterator.h"<br>
+<br>
+#define DEBUG_TYPE "loop-vectorize"<br>
+<br>
+using namespace llvm;<br>
+<br>
+// Class that is used to build the plain CFG for the incoming IR.<br>
+class PlainCFGBuilder {<br>
+private:<br>
+ // The outermost loop of the input loop nest considered for vectorization.<br>
+ Loop *TheLoop;<br>
+<br>
+ // Loop Info analysis.<br>
+ LoopInfo *LI;<br>
+<br>
+ // Vectorization plan that we are working on.<br>
+ VPlan &Plan;<br>
+<br>
+ // Output Top Region.<br>
+ VPRegionBlock *TopRegion = nullptr;<br>
+<br>
+ // Builder of the VPlan instruction-level representation.<br>
+ VPBuilder VPIRBuilder;<br>
+<br>
+ // NOTE: The following maps are intentionally destroyed after the plain CFG<br>
+ // construction because subsequent VPlan-to-VPlan transformation may<br>
+ // invalidate them.<br>
+ // Map incoming BasicBlocks to their newly-created VPBasicBlocks.<br>
+ DenseMap<BasicBlock *, VPBasicBlock *> BB2VPBB;<br>
+ // Map incoming Value definitions to their newly-created VPValues.<br>
+ DenseMap<Value *, VPValue *> IRDef2VPValue;<br>
+<br>
+ // Hold phi node's that need to be fixed once the plain CFG has been built.<br>
+ SmallVector<PHINode *, 8> PhisToFix;<br>
+<br>
+ // Utility functions.<br>
+ void setVPBBPredsFromBB(VPBasicBlock *VPBB, BasicBlock *BB);<br>
+ void fixPhiNodes();<br>
+ VPBasicBlock *getOrCreateVPBB(BasicBlock *BB);<br>
+ bool isExternalDef(Value *Val);<br>
+ VPValue *getOrCreateVPOperand(Value *IRVal);<br>
+ void createVPInstructionsForVPBB(VPBasicBlock *VPBB, BasicBlock *BB);<br>
+<br>
+public:<br>
+ PlainCFGBuilder(Loop *Lp, LoopInfo *LI, VPlan &P)<br>
+ : TheLoop(Lp), LI(LI), Plan(P) {}<br>
+<br>
+ // Build the plain CFG and return its Top Region.<br>
+ VPRegionBlock *buildPlainCFG();<br>
+};<br>
+<br>
+// Return true if \p Inst is an incoming Instruction to be ignored in the VPlan<br>
+// representation.<br>
+static bool isInstructionToIgnore(Instruction *Inst) {<br>
+ return isa<BranchInst>(Inst);<br>
+}<br>
+<br>
+// Set predecessors of \p VPBB in the same order as they are in \p BB. \p VPBB<br>
+// must have no predecessors.<br>
+void PlainCFGBuilder::setVPBBPredsFromBB(VPBasicBlock *VPBB, BasicBlock *BB) {<br>
+ SmallVector<VPBlockBase *, 8> VPBBPreds;<br>
+ // Collect VPBB predecessors.<br>
+ for (BasicBlock *Pred : predecessors(BB))<br>
+ VPBBPreds.push_back(getOrCreateVPBB(Pred));<br>
+<br>
+ VPBB->setPredecessors(VPBBPreds);<br>
+}<br>
+<br>
+// Add operands to VPInstructions representing phi nodes from the input IR.<br>
+void PlainCFGBuilder::fixPhiNodes() {<br>
+ for (auto *Phi : PhisToFix) {<br>
+ assert(IRDef2VPValue.count(Phi) && "Missing VPInstruction for PHINode.");<br>
+ VPValue *VPVal = IRDef2VPValue[Phi];<br>
+ assert(isa<VPInstruction>(VPVal) && "Expected VPInstruction for phi node.");<br>
+ auto *VPPhi = cast<VPInstruction>(VPVal);<br>
+ assert(VPPhi->getNumOperands() == 0 &&<br>
+ "Expected VPInstruction with no operands.");<br>
+<br>
+ for (Value *Op : Phi->operands())<br>
+ VPPhi->addOperand(getOrCreateVPOperand(Op));<br>
+ }<br>
+}<br>
+<br>
+// Create a new empty VPBasicBlock for an incoming BasicBlock or retrieve an<br>
+// existing one if it was already created.<br>
+VPBasicBlock *PlainCFGBuilder::getOrCreateVPBB(BasicBlock *BB) {<br>
+ auto BlockIt = BB2VPBB.find(BB);<br>
+ if (BlockIt != BB2VPBB.end())<br>
+ // Retrieve existing VPBB.<br>
+ return BlockIt->second;<br>
+<br>
+ // Create new VPBB.<br>
+ DEBUG(dbgs() << "Creating VPBasicBlock for " << BB->getName() << "\n");<br>
+ VPBasicBlock *VPBB = new VPBasicBlock(BB->getName());<br>
+ BB2VPBB[BB] = VPBB;<br>
+ VPBB->setParent(TopRegion);<br>
+ return VPBB;<br>
+}<br>
+<br>
+// Return true if \p Val is considered an external definition. An external<br>
+// definition is either:<br>
+// 1. A Value that is not an Instruction. This will be refined in the future.<br>
+// 2. An Instruction that is outside of the CFG snippet represented in VPlan,<br>
+// i.e., is not part of: a) the loop nest, b) outermost loop PH and, c)<br>
+// outermost loop exits.<br>
+bool PlainCFGBuilder::isExternalDef(Value *Val) {<br>
+ // All the Values that are not Instructions are considered external<br>
+ // definitions for now.<br>
+ Instruction *Inst = dyn_cast<Instruction>(Val);<br>
+ if (!Inst)<br>
+ return true;<br>
+<br>
+ BasicBlock *InstParent = Inst->getParent();<br>
+ assert(InstParent && "Expected instruction parent.");<br>
+<br>
+ // Check whether Instruction definition is in loop PH.<br>
+ BasicBlock *PH = TheLoop->getLoopPreheader();<br>
+ assert(PH && "Expected loop pre-header.");<br>
+<br>
+ if (InstParent == PH)<br>
+ // Instruction definition is in outermost loop PH.<br>
+ return false;<br>
+<br>
+ // Check whether Instruction definition is in the loop exit.<br>
+ BasicBlock *Exit = TheLoop->getUniqueExitBlock();<br>
+ assert(Exit && "Expected loop with single exit.");<br>
+ if (InstParent == Exit) {<br>
+ // Instruction definition is in outermost loop exit.<br>
+ return false;<br>
+ }<br>
+<br>
+ // Check whether Instruction definition is in loop body.<br>
+ return !TheLoop->contains(Inst);<br>
+}<br>
+<br>
+// Create a new VPValue or retrieve an existing one for the Instruction's<br>
+// operand \p IRVal. This function must only be used to create/retrieve VPValues<br>
+// for *Instruction's operands* and not to create regular VPInstruction's. For<br>
+// the latter, please, look at 'createVPInstructionsForVPBB'.<br>
+VPValue *PlainCFGBuilder::getOrCreateVPOperand(Value *IRVal) {<br>
+ auto VPValIt = IRDef2VPValue.find(IRVal);<br>
+ if (VPValIt != IRDef2VPValue.end())<br>
+ // Operand has an associated VPInstruction or VPValue that was previously<br>
+ // created.<br>
+ return VPValIt->second;<br>
+<br>
+ // Operand doesn't have a previously created VPInstruction/VPValue. This<br>
+ // means that operand is:<br>
+ // A) a definition external to VPlan,<br>
+ // B) any other Value without specific representation in VPlan.<br>
+ // For now, we use VPValue to represent A and B and classify both as external<br>
+ // definitions. We may introduce specific VPValue subclasses for them in the<br>
+ // future.<br>
+ assert(isExternalDef(IRVal) && "Expected external definition as operand.");<br>
+<br>
+ // A and B: Create VPValue and add it to the pool of external definitions and<br>
+ // to the Value->VPValue map.<br>
+ VPValue *NewVPVal = new VPValue(IRVal);<br>
+ Plan.addExternalDef(NewVPVal);<br>
+ IRDef2VPValue[IRVal] = NewVPVal;<br>
+ return NewVPVal;<br>
+}<br>
+<br>
+// Create new VPInstructions in a VPBasicBlock, given its BasicBlock<br>
+// counterpart. This function must be invoked in RPO so that the operands of a<br>
+// VPInstruction in \p BB have been visited before (except for Phi nodes).<br>
+void PlainCFGBuilder::createVPInstructionsForVPBB(VPBasicBlock *VPBB,<br>
+ BasicBlock *BB) {<br>
+ VPIRBuilder.setInsertPoint(VPBB);<br>
+ for (Instruction &InstRef : *BB) {<br>
+ Instruction *Inst = &InstRef;<br>
+ if (isInstructionToIgnore(Inst))<br>
+ continue;<br>
+<br>
+ // There should't be any VPValue for Inst at this point. Otherwise, we<br>
+ // visited Inst when we shouldn't, breaking the RPO traversal order.<br>
+ assert(!IRDef2VPValue.count(Inst) &&<br>
+ "Instruction shouldn't have been visited.");<br>
+<br>
+ VPInstruction *NewVPInst;<br>
+ if (PHINode *Phi = dyn_cast<PHINode>(Inst)) {<br>
+ // Phi node's operands may have not been visited at this point. We create<br>
+ // an empty VPInstruction that we will fix once the whole plain CFG has<br>
+ // been built.<br>
+ NewVPInst = cast<VPInstruction>(VPIRBuilder.createNaryOp(<br>
+ Inst->getOpcode(), {} /*No operands*/, Inst));<br>
+ PhisToFix.push_back(Phi);<br>
+ } else {<br>
+ // Translate LLVM-IR operands into VPValue operands and set them in the<br>
+ // new VPInstruction.<br>
+ SmallVector<VPValue *, 4> VPOperands;<br>
+ for (Value *Op : Inst->operands())<br>
+ VPOperands.push_back(getOrCreateVPOperand(Op));<br>
+<br>
+ // Build VPInstruction for any arbitraty Instruction without specific<br>
+ // representation in VPlan.<br>
+ NewVPInst = cast<VPInstruction>(<br>
+ VPIRBuilder.createNaryOp(Inst->getOpcode(), VPOperands, Inst));<br>
+ }<br>
+<br>
+ IRDef2VPValue[Inst] = NewVPInst;<br>
+ }<br>
+}<br>
+<br>
+// Main interface to build the plain CFG.<br>
+VPRegionBlock *PlainCFGBuilder::buildPlainCFG() {<br>
+ // 1. Create the Top Region. It will be the parent of all VPBBs.<br>
+ TopRegion = new VPRegionBlock("TopRegion", false /*isReplicator*/);<br>
+<br>
+ // 2. Scan the body of the loop in a topological order to visit each basic<br>
+ // block after having visited its predecessor basic blocks. Create a VPBB for<br>
+ // each BB and link it to its successor and predecessor VPBBs. Note that<br>
+ // predecessors must be set in the same order as they are in the incomming IR.<br>
+ // Otherwise, there might be problems with existing phi nodes and algorithm<br>
+ // based on predecessors traversal.<br>
+<br>
+ // Loop PH needs to be explicitly visited since it's not taken into account by<br>
+ // LoopBlocksDFS.<br>
+ BasicBlock *PreheaderBB = TheLoop->getLoopPreheader();<br>
+ assert((PreheaderBB->getTerminator()->getNumSuccessors() == 1) &&<br>
+ "Unexpected loop preheader");<br>
+ VPBasicBlock *PreheaderVPBB = getOrCreateVPBB(PreheaderBB);<br>
+ createVPInstructionsForVPBB(PreheaderVPBB, PreheaderBB);<br>
+ // Create empty VPBB for Loop H so that we can link PH->H.<br>
+ VPBlockBase *HeaderVPBB = getOrCreateVPBB(TheLoop->getHeader());<br>
+ // Preheader's predecessors will be set during the loop RPO traversal below.<br>
+ PreheaderVPBB->setOneSuccessor(HeaderVPBB);<br>
+<br>
+ LoopBlocksRPO RPO(TheLoop);<br>
+ RPO.perform(LI);<br>
+<br>
+ for (BasicBlock *BB : RPO) {<br>
+ // Create or retrieve the VPBasicBlock for this BB and create its<br>
+ // VPInstructions.<br>
+ VPBasicBlock *VPBB = getOrCreateVPBB(BB);<br>
+ createVPInstructionsForVPBB(VPBB, BB);<br>
+<br>
+ // Set VPBB successors. We create empty VPBBs for successors if they don't<br>
+ // exist already. Recipes will be created when the successor is visited<br>
+ // during the RPO traversal.<br>
+ TerminatorInst *TI = BB->getTerminator();<br>
+ assert(TI && "Terminator expected.");<br>
+ unsigned NumSuccs = TI->getNumSuccessors();<br>
+<br>
+ if (NumSuccs == 1) {<br>
+ VPBasicBlock *SuccVPBB = getOrCreateVPBB(TI->getSuccessor(0));<br>
+ assert(SuccVPBB && "VPBB Successor not found.");<br>
+ VPBB->setOneSuccessor(SuccVPBB);<br>
+ } else if (NumSuccs == 2) {<br>
+ VPBasicBlock *SuccVPBB0 = getOrCreateVPBB(TI->getSuccessor(0));<br>
+ assert(SuccVPBB0 && "Successor 0 not found.");<br>
+ VPBasicBlock *SuccVPBB1 = getOrCreateVPBB(TI->getSuccessor(1));<br>
+ assert(SuccVPBB1 && "Successor 1 not found.");<br>
+ VPBB->setTwoSuccessors(SuccVPBB0, SuccVPBB1);<br>
+ } else<br>
+ llvm_unreachable("Number of successors not supported.");<br>
+<br>
+ // Set VPBB predecessors in the same order as they are in the incoming BB.<br>
+ setVPBBPredsFromBB(VPBB, BB);<br>
+ }<br>
+<br>
+ // 3. Process outermost loop exit. We created an empty VPBB for the loop<br>
+ // single exit BB during the RPO traversal of the loop body but Instructions<br>
+ // weren't visited because it's not part of the the loop.<br>
+ BasicBlock *LoopExitBB = TheLoop->getUniqueExitBlock();<br>
+ assert(LoopExitBB && "Loops with multiple exits are not supported.");<br>
+ VPBasicBlock *LoopExitVPBB = BB2VPBB[LoopExitBB];<br>
+ createVPInstructionsForVPBB(LoopExitVPBB, LoopExitBB);<br>
+ // Loop exit was already set as successor of the loop exiting BB.<br>
+ // We only set its predecessor VPBB now.<br>
+ setVPBBPredsFromBB(LoopExitVPBB, LoopExitBB);<br>
+<br>
+ // 4. The whole CFG has been built at this point so all the input Values must<br>
+ // have a VPlan couterpart. Fix VPlan phi nodes by adding their corresponding<br>
+ // VPlan operands.<br>
+ fixPhiNodes();<br>
+<br>
+ // 5. Final Top Region setup. Set outermost loop pre-header and single exit as<br>
+ // Top Region entry and exit.<br>
+ TopRegion->setEntry(PreheaderVPBB);<br>
+ TopRegion->setExit(LoopExitVPBB);<br>
+ return TopRegion;<br>
+}<br>
+<br>
+// Public interface to build a H-CFG.<br>
+void VPlanHCFGBuilder::buildHierarchicalCFG(VPlan &Plan) {<br>
+ // Build Top Region enclosing the plain CFG and set it as VPlan entry.<br>
+ PlainCFGBuilder PCFGBuilder(TheLoop, LI, Plan);<br>
+ VPRegionBlock *TopRegion = PCFGBuilder.buildPlainCFG();<br>
+ Plan.setEntry(TopRegion);<br>
+ DEBUG(Plan.setName("HCFGBuilder: Plain CFG\n"); dbgs() << Plan);<br>
+<br>
+ Verifier.verifyHierarchicalCFG(TopRegion);<br>
+}<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp<br>
------------------------------------------------------------------------------<br>
svn:eol-style = native<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp<br>
------------------------------------------------------------------------------<br>
svn:keywords = Author Date Id Rev URL<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp<br>
------------------------------------------------------------------------------<br>
svn:mime-type = text/plain<br>
<br>
Added: llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.h?rev=332860&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.h?rev=332860&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.h (added)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.h Mon May 21 11:14:23 2018<br>
@@ -0,0 +1,55 @@<br>
+//===-- VPlanHCFGBuilder.h --------------------------------------*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+///<br>
+/// \file<br>
+/// This file defines the VPlanHCFGBuilder class which contains the public<br>
+/// interface (buildHierarchicalCFG) to build a VPlan-based Hierarchical CFG<br>
+/// (H-CFG) for an incoming IR.<br>
+///<br>
+/// A H-CFG in VPlan is a control-flow graph whose nodes are VPBasicBlocks<br>
+/// and/or VPRegionBlocks (i.e., other H-CFGs). The outermost H-CFG of a VPlan<br>
+/// consists of a VPRegionBlock, denoted Top Region, which encloses any other<br>
+/// VPBlockBase in the H-CFG. This guarantees that any VPBlockBase in the H-CFG<br>
+/// other than the Top Region will have a parent VPRegionBlock and allows us<br>
+/// to easily add more nodes before/after the main vector loop (such as the<br>
+/// reduction epilogue).<br>
+///<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#ifndef LLVM_TRANSFORMS_VECTORIZE_VPLAN_VPLANHCFGBUILDER_H<br>
+#define LLVM_TRANSFORMS_VECTORIZE_VPLAN_VPLANHCFGBUILDER_H<br>
+<br>
+#include "VPlan.h"<br>
+#include "VPlanVerifier.h"<br>
+<br>
+namespace llvm {<br>
+<br>
+class Loop;<br>
+<br>
+/// Main class to build the VPlan H-CFG for an incoming IR.<br>
+class VPlanHCFGBuilder {<br>
+private:<br>
+ // The outermost loop of the input loop nest considered for vectorization.<br>
+ Loop *TheLoop;<br>
+<br>
+ // Loop Info analysis.<br>
+ LoopInfo *LI;<br>
+<br>
+ // VPlan verifier utility.<br>
+ VPlanVerifier Verifier;<br>
+<br>
+public:<br>
+ VPlanHCFGBuilder(Loop *Lp, LoopInfo *LI) : TheLoop(Lp), LI(LI) {}<br>
+<br>
+ /// Build H-CFG for TheLoop and update \p Plan accordingly.<br>
+ void buildHierarchicalCFG(VPlan &Plan);<br>
+};<br>
+} // namespace llvm<br>
+<br>
+#endif // LLVM_TRANSFORMS_VECTORIZE_VPLAN_VPLANHCFGBUILDER_H<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.h<br>
------------------------------------------------------------------------------<br>
svn:eol-style = native<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.h<br>
------------------------------------------------------------------------------<br>
svn:keywords = Author Date Id Rev URL<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanHCFGBuilder.h<br>
------------------------------------------------------------------------------<br>
svn:mime-type = text/plain<br>
<br>
Modified: llvm/trunk/lib/Transforms/Vectorize/VPlanValue.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanValue.h?rev=332860&r1=332859&r2=332860&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanValue.h?rev=332860&r1=332859&r2=332860&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/VPlanValue.h (original)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/VPlanValue.h Mon May 21 11:14:23 2018<br>
@@ -37,13 +37,34 @@ class VPUser;<br>
// coming from the input IR, instructions which VPlan will generate if executed<br>
// and live-outs which the VPlan will need to fix accordingly.<br>
class VPValue {<br>
+ friend class VPBuilder;<br>
private:<br>
const unsigned char SubclassID; ///< Subclass identifier (for isa/dyn_cast).<br>
<br>
SmallVector<VPUser *, 1> Users;<br>
<br>
protected:<br>
- VPValue(const unsigned char SC) : SubclassID(SC) {}<br>
+ // Hold the underlying Value, if any, attached to this VPValue.<o:p></o:p></p>
<p class="MsoNormal">+ Value *UnderlyingVal;<br>
+<br>
+ VPValue(const unsigned char SC, Value *UV = nullptr)<br>
+ : SubclassID(SC), UnderlyingVal(UV) {}<br>
+<br>
+ // DESIGN PRINCIPLE: Access to the underlying IR must be strictly limited to<br>
+ // the front-end and back-end of VPlan so that the middle-end is as<br>
+ // independent as possible of the underlying IR. We grant access to the<br>
+ // underlying IR using friendship. In that way, we should be able to use VPlan<br>
+ // for multiple underlying IRs (Polly?) by providing a new VPlan front-end,<br>
+ // back-end and analysis information for the new IR.<br>
+<br>
+ /// Return the underlying Value attached to this VPValue.<br>
+ Value *getUnderlyingValue() { return UnderlyingVal; }<br>
+<br>
+ // Set \p Val as the underlying Value of this VPValue.<br>
+ void setUnderlyingValue(Value *Val) {<br>
+ assert(!UnderlyingVal && "Underlying Value is already set.");<br>
+ UnderlyingVal = Val;<br>
+ }<br>
<br>
public:<br>
/// An enumeration for keeping track of the concrete subclass of VPValue that<br>
@@ -52,7 +73,7 @@ public:<br>
/// type identification.<br>
enum { VPValueSC, VPUserSC, VPInstructionSC };<br>
<br>
- VPValue() : SubclassID(VPValueSC) {}<br>
+ VPValue(Value *UV = nullptr) : VPValue(VPValueSC, UV) {}<br>
VPValue(const VPValue &) = delete;<br>
VPValue &operator=(const VPValue &) = delete;<br>
<br>
@@ -94,11 +115,6 @@ class VPUser : public VPValue {<br>
private:<br>
SmallVector<VPValue *, 2> Operands;<br>
<br>
- void addOperand(VPValue *Operand) {<br>
- Operands.push_back(Operand);<br>
- Operand->addUser(*this);<br>
- }<br>
-<br>
protected:<br>
VPUser(const unsigned char SC) : VPValue(SC) {}<br>
VPUser(const unsigned char SC, ArrayRef<VPValue *> Operands) : VPValue(SC) {<br>
@@ -120,6 +136,11 @@ public:<br>
V->getVPValueID() <= VPInstructionSC;<br>
}<br>
<br>
+ void addOperand(VPValue *Operand) {<br>
+ Operands.push_back(Operand);<br>
+ Operand->addUser(*this);<br>
+ }<br>
+<br>
unsigned getNumOperands() const { return Operands.size(); }<br>
inline VPValue *getOperand(unsigned N) const {<br>
assert(N < Operands.size() && "Operand index out of bounds");<br>
<br>
Added: llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.cpp?rev=332860&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.cpp?rev=332860&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.cpp (added)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.cpp Mon May 21 11:14:23 2018<br>
@@ -0,0 +1,126 @@<br>
+//===-- VPlanVerifier.cpp -------------------------------------------------===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+///<br>
+/// \file<br>
+/// This file defines the class VPlanVerifier, which contains utility functions<br>
+/// to check the consistency and invariants of a VPlan.<br>
+///<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include "VPlanVerifier.h"<br>
+#include "llvm/ADT/DepthFirstIterator.h"<br>
+<br>
+#define DEBUG_TYPE "loop-vectorize"<br>
+<br>
+using namespace llvm;<br>
+<br>
+static cl::opt<bool> EnableHCFGVerifier("vplan-verify-hcfg", cl::init(false),<br>
+ cl::Hidden,<br>
+ cl::desc("Verify VPlan H-CFG."));<br>
+<br>
+/// Utility function that checks whether \p VPBlockVec has duplicate<br>
+/// VPBlockBases.<br>
+static bool hasDuplicates(const SmallVectorImpl<VPBlockBase *> &VPBlockVec) {<br>
+ SmallDenseSet<const VPBlockBase *, 8> VPBlockSet;<br>
+ for (const auto *Block : VPBlockVec) {<br>
+ if (VPBlockSet.count(Block))<br>
+ return true;<br>
+ VPBlockSet.insert(Block);<br>
+ }<br>
+ return false;<br>
+}<br>
+<br>
+/// Helper function that verifies the CFG invariants of the VPBlockBases within<br>
+/// \p Region. Checks in this function are generic for VPBlockBases. They are<br>
+/// not specific for VPBasicBlocks or VPRegionBlocks.<br>
+static void verifyBlocksInRegion(const VPRegionBlock *Region) {<br>
+ for (const VPBlockBase *VPB :<br>
+ make_range(df_iterator<const VPBlockBase *>::begin(Region->getEntry()),<br>
+ df_iterator<const VPBlockBase *>::end(Region->getExit()))) {<br>
+ // Check block's parent.<br>
+ assert(VPB->getParent() == Region && "VPBlockBase has wrong parent");<br>
+<br>
+ // Check block's successors.<br>
+ const auto &Successors = VPB->getSuccessors();<br>
+ // There must be only one instance of a successor in block's successor list.<br>
+ // TODO: This won't work for switch statements.<br>
+ assert(!hasDuplicates(Successors) &&<br>
+ "Multiple instances of the same successor.");<br>
+ (void)hasDuplicates;<br>
+<br>
+ for (const VPBlockBase *Succ : Successors) {<br>
+ // There must be a bi-directional link between block and successor.<br>
+ const auto &SuccPreds = Succ->getPredecessors();<br>
+ assert(std::find(SuccPreds.begin(), SuccPreds.end(), VPB) !=<br>
+ SuccPreds.end() &&<br>
+ "Missing predecessor link.");<br>
+ (void)SuccPreds;<br>
+ }<br>
+<br>
+ // Check block's predecessors.<br>
+ const auto &Predecessors = VPB->getPredecessors();<br>
+ // There must be only one instance of a predecessor in block's predecessor<br>
+ // list.<br>
+ // TODO: This won't work for switch statements.<br>
+ assert(!hasDuplicates(Predecessors) &&<br>
+ "Multiple instances of the same predecessor.");<br>
+<br>
+ for (const VPBlockBase *Pred : Predecessors) {<br>
+ // Block and predecessor must be inside the same region.<br>
+ assert(Pred->getParent() == VPB->getParent() &&<br>
+ "Predecessor is not in the same region.");<br>
+<br>
+ // There must be a bi-directional link between block and predecessor.<br>
+ const auto &PredSuccs = Pred->getSuccessors();<br>
+ assert(std::find(PredSuccs.begin(), PredSuccs.end(), VPB) !=<br>
+ PredSuccs.end() &&<br>
+ "Missing successor link.");<br>
+ (void)PredSuccs;<br>
+ }<br>
+ }<br>
+}<br>
+<br>
+/// Verify the CFG invariants of VPRegionBlock \p Region and its nested<br>
+/// VPBlockBases. Do not recurse inside nested VPRegionBlocks.<br>
+static void verifyRegion(const VPRegionBlock *Region) {<br>
+ const VPBlockBase *Entry = Region->getEntry();<br>
+ const VPBlockBase *Exit = Region->getExit();<br>
+<br>
+ // Entry and Exit shouldn't have any predecessor/successor, respectively.<br>
+ assert(!Entry->getNumPredecessors() && "Region entry has predecessors.");<br>
+ assert(!Exit->getNumSuccessors() && "Region exit has successors.");<br>
+ (void)Entry;<br>
+ (void)Exit;<br>
+<br>
+ verifyBlocksInRegion(Region);<br>
+}<br>
+<br>
+/// Verify the CFG invariants of VPRegionBlock \p Region and its nested<br>
+/// VPBlockBases. Recurse inside nested VPRegionBlocks.<br>
+static void verifyRegionRec(const VPRegionBlock *Region) {<br>
+ verifyRegion(Region);<br>
+<br>
+ // Recurse inside nested regions.<br>
+ for (const VPBlockBase *VPB :<br>
+ make_range(df_iterator<const VPBlockBase *>::begin(Region->getEntry()),<br>
+ df_iterator<const VPBlockBase *>::end(Region->getExit()))) {<br>
+ if (const auto *SubRegion = dyn_cast<VPRegionBlock>(VPB))<br>
+ verifyRegionRec(SubRegion);<br>
+ }<br>
+}<br>
+<br>
+void VPlanVerifier::verifyHierarchicalCFG(<br>
+ const VPRegionBlock *TopRegion) const {<br>
+ if (!EnableHCFGVerifier)<br>
+ return;<br>
+<br>
+ DEBUG(dbgs() << "Verifying VPlan H-CFG.\n");<br>
+ assert(!TopRegion->getParent() && "VPlan Top Region should have no parent.");<br>
+ verifyRegionRec(TopRegion);<br>
+}<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.cpp<br>
------------------------------------------------------------------------------<br>
svn:eol-style = native<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.cpp<br>
------------------------------------------------------------------------------<br>
svn:keywords = Author Date Id Rev URL<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.cpp<br>
------------------------------------------------------------------------------<br>
svn:mime-type = text/plain<br>
<br>
Added: llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.h?rev=332860&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.h?rev=332860&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.h (added)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.h Mon May 21 11:14:23 2018<br>
@@ -0,0 +1,44 @@<br>
+//===-- VPlanVerifier.h -----------------------------------------*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+///<br>
+/// \file<br>
+/// This file declares the class VPlanVerifier, which contains utility functions<br>
+/// to check the consistency of a VPlan. This includes the following kinds of<br>
+/// invariants:<br>
+///<br>
+/// 1. Region/Block invariants:<br>
+/// - Region's entry/exit block must have no predecessors/successors,<br>
+/// respectively.<br>
+/// - Block's parent must be the region immediately containing the block.<br>
+/// - Linked blocks must have a bi-directional link (successor/predecessor).<br>
+/// - All predecessors/successors of a block must belong to the same region.<br>
+/// - Blocks must have no duplicated successor/predecessor.<br>
+///<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#ifndef LLVM_TRANSFORMS_VECTORIZE_VPLANVERIFIER_H<br>
+#define LLVM_TRANSFORMS_VECTORIZE_VPLANVERIFIER_H<br>
+<br>
+#include "VPlan.h"<br>
+<br>
+namespace llvm {<br>
+<br>
+/// Class with utility functions that can be used to check the consistency and<br>
+/// invariants of a VPlan, including the components of its H-CFG.<br>
+class VPlanVerifier {<br>
+public:<br>
+ /// Verify the invariants of the H-CFG starting from \p TopRegion. The<br>
+ /// verification process comprises the following steps:<br>
+ /// 1. Region/Block verification: Check the Region/Block verification<br>
+ /// invariants for every region in the H-CFG.<br>
+ void verifyHierarchicalCFG(const VPRegionBlock *TopRegion) const;<br>
+};<br>
+} // namespace llvm<br>
+<br>
+#endif //LLVM_TRANSFORMS_VECTORIZE_VPLANVERIFIER_H<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.h<br>
------------------------------------------------------------------------------<br>
svn:eol-style = native<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.h<br>
------------------------------------------------------------------------------<br>
svn:keywords = Author Date Id Rev URL<br>
<br>
Propchange: llvm/trunk/lib/Transforms/Vectorize/VPlanVerifier.h<br>
------------------------------------------------------------------------------<br>
svn:mime-type = text/plain<br>
<br>
Added: llvm/trunk/test/Transforms/LoopVectorize/vplan_hcfg_stress_test.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/LoopVectorize/vplan_hcfg_stress_test.ll?rev=332860&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/LoopVectorize/vplan_hcfg_stress_test.ll?rev=332860&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/Transforms/LoopVectorize/vplan_hcfg_stress_test.ll (added)<br>
+++ llvm/trunk/test/Transforms/LoopVectorize/vplan_hcfg_stress_test.ll Mon May 21 11:14:23 2018<br>
@@ -0,0 +1,51 @@<br>
+; RUN: opt < %s -loop-vectorize -enable-vplan-native-path -vplan-build-stress-test -vplan-verify-hcfg -debug-only=loop-vectorize -disable-output 2>&1 | FileCheck %s -check-prefix=VERIFIER<br>
+; RUN: opt < %s -loop-vectorize -enable-vplan-native-path -vplan-build-stress-test -debug-only=loop-vectorize -disable-output 2>&1 | FileCheck %s -check-prefix=NO-VERIFIER -allow-empty<br>
+; REQUIRES: asserts<br>
+<br>
+; Verify that the stress testing flag for the VPlan H-CFG builder works as<br>
+; expected with and without enabling the VPlan H-CFG Verifier.<br>
+<br>
+; VERIFIER: Verifying VPlan H-CFG.<br>
+; NO-VERIFIER-NOT: Verifying VPlan H-CFG.<br>
+<br>
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"<br>
+<br>
+define void @foo(i32* nocapture %a, i32* nocapture readonly %b, i32 %N, i32 %M) {<br>
+entry:<br>
+ %cmp32 = icmp sgt i32 %N, 0<br>
+ br i1 %cmp32, label %<a href="http://outer.ph" target="_blank">outer.ph</a>, label %for.end15<br>
+<br>
+<a href="http://outer.ph" target="_blank">outer.ph</a>:<br>
+ %cmp230 = icmp sgt i32 %M, 0<br>
+ %0 = sext i32 %M to i64<br>
+ %wide.trip.count = zext i32 %M to i64<br>
+ %wide.trip.count38 = zext i32 %N to i64<br>
+ br label %outer.body<br>
+<br>
+outer.body:<br>
+ %indvars.iv35 = phi i64 [ 0, %<a href="http://outer.ph" target="_blank">outer.ph</a> ], [ %indvars.iv.next36, %outer.inc ]<br>
+ br i1 %cmp230, label %<a href="http://inner.ph" target="_blank">inner.ph</a>, label %outer.inc<br>
+<br>
+<a href="http://inner.ph" target="_blank">inner.ph</a>:<br>
+ %1 = mul nsw i64 %indvars.iv35, %0<br>
+ br label %inner.body<br>
+<br>
+inner.body:<br>
+ %indvars.iv = phi i64 [ 0, %<a href="http://inner.ph" target="_blank">inner.ph</a> ], [ %indvars.iv.next, %inner.body ]<br>
+ %2 = add nsw i64 %indvars.iv, %1<br>
+ %arrayidx = getelementptr inbounds i32, i32* %b, i64 %2<br>
+ %3 = load i32, i32* %arrayidx, align 4<br>
+ %arrayidx12 = getelementptr inbounds i32, i32* %a, i64 %2<br>
+ store i32 %3, i32* %arrayidx12, align 4<br>
+ %indvars.iv.next = add nuw nsw i64 %indvars.iv, 1<br>
+ %exitcond = icmp eq i64 %indvars.iv.next, %wide.trip.count<br>
+ br i1 %exitcond, label %outer.inc, label %inner.body<br>
+<br>
+outer.inc:<br>
+ %indvars.iv.next36 = add nuw nsw i64 %indvars.iv35, 1<br>
+ %exitcond39 = icmp eq i64 %indvars.iv.next36, %wide.trip.count38<br>
+ br i1 %exitcond39, label %for.end15, label %outer.body<br>
+<br>
+for.end15:<br>
+ ret void<br>
+}<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>