<div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div>Sorry David:</div><div><br></div><div>I make a mistake in yesterday explanation.</div><div><br></div><div>Here is some data I collect during building Tablegen, the data is from x86 target</div><div><br></div><div>llvm[3]: Building X86.td instruction information with tblgen</div><div><span style="line-height: 1.7;">NumTPNAllocCur: 120754, NumTPNAllocAccu: 121469 <-- it indicates only 121469-120754=715 heap alloc TreePatternNodes get freed during processing (~0.5%)</span></div><div>NumTPAllocCur: 9488, NumTPAllocAccu: 9488 <-- in fact <b>NO</b> heap alloc TreePatterns get freed during processing</div><div>NumTPNAllocCur: 0, NumTPNDel: 121469</div><div>NumTPAllocCur: 0, NumTPDel: 9488</div><div>TPN dtorNum: 121469, TP dtorNum: 16821 <-- the diff "<span style="line-height: 23.7999992370605px;">NumTPAllocAccu"</span><span style="line-height: 23.7999992370605px;">-"</span><span style="line-height: 23.7999992370605px;">TPN dtorNum"</span><span style="line-height: 23.7999992370605px;">=0 indicates that all TreePatternNodes are heap allocated</span></div><div>TPN ctorNum: 121469, TP ctorNum: 16821 <-- the diff "<span style="line-height: 23.7999992370605px;">TP ctorNum"</span><span style="line-height: 23.7999992370605px;">-"</span><span style="line-height: 23.7999992370605px;">NumTPDel"</span><span style="line-height: 23.7999992370605px;">=7333 indicates that 7333 TreePatterns are placed into stack!</span></div><div><br></div><div><div>llvm[3]: Building X86.td DAG instruction selector implementation with tblgen</div><div>NumTPNAllocCur: 180382, NumTPNAllocAccu: 197494 <-- 17112 TreePatternNodes (~8%) get freed</div><div>NumTPAllocCur: 9488, NumTPAllocAccu: 9488<span style="line-height: 23.7999992370605px;"> <-- </span><b style="line-height: 23.7999992370605px;">NO</b><span style="line-height: 23.7999992370605px;"> heap alloc TreePatterns get freed during processing</span></div><div>NumTPNAllocCur: 0, NumTPNDel: 197494 <-- "<span style="line-height: 23.7999992370605px;">NumTPNDel" == "</span><span style="line-height: 23.7999992370605px;">TPN ctorNum": all TreePatternNodes all heap allocated</span></div><div>NumTPAllocCur: 0, NumTPDel: 9488</div><div>TPN dtorNum: 197494, TP dtorNum: 16821 <-- the diff <span style="line-height: 23.7999992370605px;">16821-</span><span style="line-height: 23.7999992370605px;">9488=</span><span style="line-height: 23.7999992370605px;">7333 indicates that 7333 TreePatterns are placed into stack!</span></div><div>TPN ctorNum: 197494, TP ctorNum: 16821</div></div><div><br></div><div>So more than 43% TreePatterns are stack residue for both processing.</div><div><br></div><div>And other target also gives similar data, for instance AMDGPU</div><div><br></div><div><div>llvm[3]: Building AMDGPU.td instruction information with tblgen</div><div>NumTPNAllocCur: 28248, NumTPNAllocAccu: 28348 <-- 100 TreePatternNodes (~0.3%) get freed</div><div>NumTPAllocCur: 2422, NumTPAllocAccu: 2422<span style="line-height: 23.7999992370605px;"> <-- </span><b style="line-height: 23.7999992370605px;">NO</b><span style="line-height: 23.7999992370605px;"> heap alloc TreePatterns get freed during processing</span></div><div>NumTPNAllocCur: 0, NumTPNDel: 28348</div><div>NumTPAllocCur: 0, NumTPDel: 2422</div><div>TPN dtorNum: 28348, TP dtorNum: 3792</div><div>TPN ctorNum: 28348, TP ctorNum: 3792 <-- 1370 TreePatterns are inside stack (>36%)</div><div>llvm[3]: Building AMDGPU.td DAG instruction selector implementation with tblgen</div><div>NumTPNAllocCur: 44076, NumTPNAllocAccu: 47030 <-- 2954 TreePatternNodes (~6%) get freed</div><div>NumTPAllocCur: 2422, NumTPAllocAccu: 2422<span style="line-height: 23.7999992370605px;"> <-- </span><b style="line-height: 23.7999992370605px;">NO</b><span style="line-height: 23.7999992370605px;"> heap alloc TreePatterns get freed during processing</span></div><div>NumTPNAllocCur: 0, NumTPNDel: 47030</div><div>NumTPAllocCur: 0, NumTPDel: 2422</div><div>TPN dtorNum: 47030, TP dtorNum: 3792</div><div>TPN ctorNum: 47030, TP ctorNum: 3792 <-- 1370 <span style="line-height: 23.7999992370605px;">TreePatterns are inside stack (>36%)</span></div></div><div><br></div><div><span style="line-height: 1.7;">There are several functions in CodeGenDAGPatterns.cpp will allocate TreePatternNodes & TreePatterns from heap.</span></div><div>ParsePatternFragments (5 spots allocate TreePatternNodes)</div><div>ParsePatterns (2 spots allocate TreePatternNodes, 2 spots allocate TreePatterns)</div><div>CombineChildVariants (1 spot allocate TreePatternNodes)</div><div>clone<span style="line-height: 23.7999992370605px;"> </span><span style="line-height: 23.7999992370605px;">(2 spots allocate TreePatternNodes)</span></div><div><span style="line-height: 23.7999992370605px;"><br></span></div><div>So I don't think there is any special construction path.</div><div><br></div><div>Regards</div><div>Hui Wu</div><div></div><div id="divNeteaseMailCard"></div><br>ÔÚ 2014-11-12 06:33:37£¬"David Blaikie" <dblaikie@gmail.com> дµÀ£º<br> <blockquote id="isReplyContent" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Nov 10, 2014 at 11:36 PM, wuhui1973 <span dir="ltr"><<a href="mailto:wuhui1973@163.com" target="_blank">wuhui1973@163.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div>Hope I can explain it clear enough :-)</div><div><br></div><div>> MemReclaimer has two containers to hold pointers of TreePatternNode & TreePattern which are allocated from heap.</div><div>> Only one global instance of <span style="line-height:23.7999992370605px">MemReclaimer</span><span style="line-height:23.7999992370605px"> is declared -- memReclaimer.</span></div><div><span style="line-height:23.7999992370605px">> operator new & operator delete are defined for </span><span style="line-height:23.7999992370605px">TreePatternNode & TreePattern </span></div><div><span style="line-height:23.7999992370605px"> new: push the new allocated object (pointer) into containers in </span><span style="line-height:23.7999992370605px">memReclaimer.</span></div><div><span style="line-height:23.7999992370605px"> delete: remove the pointer from the containers in </span><span style="line-height:23.7999992370605px">memReclaimer.</span></div><div><span style="line-height:23.7999992370605px"><br></span></div><div><span style="line-height:23.7999992370605px"> So we can </span>guarantee that objects allocated from heap can always be recorded by <span style="line-height:23.7999992370605px">memReclaimer.</span><span style="line-height:23.7999992370605px"> </span></div><div><span style="line-height:23.7999992370605px"> And no double free may incur, as some places in the source delete some pointers manually (maybe we can remove these lines</span><span style="line-height:23.7999992370605px">).</span></div><div><span style="line-height:23.7999992370605px">> So memReclaimer only keeps the objects that are leaked in current design.</span></div><div><span style="line-height:23.7999992370605px">> the author of </span><span style="line-height:23.7999992370605px">CodeGenDAGPattern </span><span style="line-height:23.7999992370605px"> create nodes, but destroy part of them (for TreePattern, the nodes </span><span style="line-height:1.7">destroyed</span><span style="line-height:23.7999992370605px"> manually account for more than 30%, </span></div></div></blockquote><div><br></div><div>Could you provide stack traces/explanation for these case where the TreePatterns are destroyed?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div><span style="line-height:23.7999992370605px"> for some target even over 50%; while for TreePatternNode only around 1%</span></div></div></blockquote><div><br>And for the TreePatternNodes?<br><br>If they're mostly just on special case construction paths, perhaps we could special case their removal from the list we could add to CodeGenDAGPattern - or, if they're close enough to construction time, we might be able to avoid adding them to such a list in the first place?<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div><span style="line-height:23.7999992370605px">) and make the rest leaked deliberately.</span></div><div>> dtor of <span style="line-height:23.7999992370605px">CodeGenDAGPattern now is the place to </span>destroy<span style="line-height:23.7999992370605px"> all not freed objects</span></div><div><span style="line-height:23.7999992370605px">> move the functionality of </span><span style="line-height:23.7999992370605px">MemReclaimer</span><span style="line-height:23.7999992370605px"> into </span><span style="line-height:23.7999992370605px">CodeGenDAGPattern </span><span style="line-height:23.7999992370605px"> is definitely feasible I think.</span><span style="font-family:arial;font-size:small;line-height:normal;color:rgb(34,34,34)"> </span></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div><br></div><div>Regards</div><div>Hui Wu</div><div><div class="h5"><div><br></div><div></div><div></div><br>At 2014-11-11 14:20:25, "David Blaikie" <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>> wrote:<br> <blockquote style="PADDING-LEFT:1ex;MARGIN:0px 0px 0px 0.8ex;BORDER-LEFT:#ccc 1px solid"><div dir="ltr">Could you describe the high level design here?<br><br>It looks like there's a static pool (CodeGenDAGPatterns.cpp::memReclaimer) of instances that's used as some kind of last-chance cleanup? Instances of TreePatternNode mostly manage their own lifetime but then if any haven't been destroyed by the time the CodeGenDAGPatterns dtor runs, the remaining elements are destroyed. This is to handle cycles, I take it?<br><br>It still seems like a bit of a GC-esque workaround to handle this case when there might be something better... but I don't know much about tablegen, perhaps there isn't.<br><br>How often are nodes destroyed by themselves? Should we just give up owning these anywhere else & move all ownership into CodeGenDAGPatterns and clean them up in CodeGenDAGPatterns' dtor? (just have a vector of unique_ptrs (maybe even a list or deque of nodes directly owned, rather than via unique_ptr), only create nodes, never destroy them, then destroy them all at the end)</div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Nov 10, 2014 at 9:58 PM, wuhui1973 <span dir="ltr"><<a href="mailto:wuhui1973@163.com" target="_blank">wuhui1973@163.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px">Hi Andrew, David and Hal:</span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px"><br></span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px">I have made a new solution for this memory leak issue, which is much simpler than previous one. </span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px"><br></span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px">I have tested it, it works well!</span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px"><br></span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px">Please have a look, and appreciate any comment.</span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px"><br></span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px">Thanks & Regards</span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px"><br></span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><span style="line-height:23.7999992370605px">Hui Wu</span></div><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><br></div></div><span title="neteasefooter"><span></span></span></div><br><br><span title="neteasefooter"><span></span></span></blockquote></div><br></div>
</blockquote></div></div></div><br><br><span title="neteasefooter"><span></span></span></blockquote></div><br></div></div>
</blockquote></div><br><br><span title="neteasefooter"><span id="netease_mail_footer"></span></span>