<div dir="ltr">On Fri, Aug 30, 2013 at 3:40 AM, Kristof Beyls <span dir="ltr"><<a href="mailto:kristof.beyls@arm.com" target="_blank" class="cremed">kristof.beyls@arm.com</a>></span> wrote:<br><div class="gmail_extra">
<div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div lang="EN-GB" link="blue" vlink="purple">
<div><p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Hi Samuel,<u></u><u></u></span></p><p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Thanks for the quick response.<u></u><u></u></span></p><p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">I agree that the best way forward would be to reduce the number of sections/symbols produced<br>
in Register.cpp.o. I’m afraid I don’t know enough about this part of clang to know how feasible<br>that is.<u></u><u></u></span></p><p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">I’m afraid I don’t really have a good target for the maximum number of sections to be produced<br>
in a single object file. I think the target should be to at least halve the number of sections of what<br>is currently produced, and hope that future changes will not be creeping up the number of<br>sections again.</span></p>
</div></div></blockquote><div><br></div><div>Just as fyi, I already submitted one change that took out ~10% of the symbols.</div><div>I have 2 more for review with similar benefit (<a href="http://llvm-reviews.chandlerc.com/D1564">http://llvm-reviews.chandlerc.com/D1564</a> and <a href="http://llvm-reviews.chandlerc.com/D1560">http://llvm-reviews.chandlerc.com/D1560</a>).</div>
<div>I think there are a couple easy ones after those, but I haven't explored them yet.</div><div>_Sam</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div lang="EN-GB" link="blue" vlink="purple"><div><p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u><u></u></span></p><p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Thanks,<u></u><u></u></span></p><p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Kristof<u></u><u></u></span></p><p class=""><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<div style="border-style:none none none solid;border-left-color:blue;border-left-width:1.5pt;padding:0cm 0cm 0cm 4pt"><div><div style="border-style:solid none none;border-top-color:rgb(181,196,223);border-top-width:1pt;padding:3pt 0cm 0cm">
<p class=""><b><span lang="EN-US" style="font-size:10pt;font-family:Tahoma,sans-serif">From:</span></b><span lang="EN-US" style="font-size:10pt;font-family:Tahoma,sans-serif"> Samuel Benzaquen [mailto:<a href="mailto:sbenza@google.com" target="_blank" class="cremed">sbenza@google.com</a>] <br>
<b>Sent:</b> 29 August 2013 23:09<br><b>To:</b> Kristof Beyls<br><b>Cc:</b> Clang Dev; Nicholas Braden; Manuel Klimek<br><b>Subject:</b> Re: 'too many sections' again when building ASTMatchers/Dynamic/Registry.cpp on Windows<u></u><u></u></span></p>
</div></div><div><div class="h5"><p class=""><u></u> <u></u></p><div><p class="">On Thu, Aug 29, 2013 at 3:49 PM, Kristof Beyls <<a href="mailto:kristof.beyls@arm.com" target="_blank" class="cremed">kristof.beyls@arm.com</a>> wrote:<u></u><u></u></p>
<div><div><blockquote style="border-style:none none none solid;border-left-color:rgb(204,204,204);border-left-width:1pt;padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"><p class="">Hi Samuel,<br><br>We've observed that in the past few days, debug-builds with Visual Studio<br>
have started failing again because too many sections are generated for<br>ASTMatchers/Dynamic/Registry.cpp.<br><br>This has been observed and discussed before in this email thread:<br><a href="http://lists.cs.uiuc.edu/pipermail/cfe-dev/2013-June/030102.html" target="_blank" class="cremed">http://lists.cs.uiuc.edu/pipermail/cfe-dev/2013-June/030102.html</a>.<br>
<br>It seems that since then, bit by bit, the number of sections for<br>Registry.cpp<br>has been creeping up. I've done a quick experiment on linux to measure how<br>the size and the number of sections in Registry.cpp.o has been increasing<br>
over<br>time. The table does show that your fix in r183768 did reduce the number of<br>sections a lot; but since then, the number of sections has been increasing<br>steadily and quite rapidly.<br><br> Size of Register.cpp.o<br>
svn id bytes sections<br>189448 27122264 28108<br>189353 27204656 28259<br>189031 26473624 27284<br>187500 26451288 26952<br>186000 21940824 23119<br>183769 19289872 20544<br>
183768 33647984 35738<br><br>You did say on the previous email thread that you had a few more ideas on<br>how<br>to reduce the number of sections in Register.cpp.o. I'm wondering if you<br>would<br>be able to share the ideas you've got?<u></u><u></u></p>
</blockquote><div><p class=""><u></u> <u></u></p></div><div><p class="">The problem is that there are a lot of AST nodes, and each node is getting a whole bunch of classes instantiated, many of them with virtual methods.<u></u><u></u></p>
</div><div><p class="">For example, there are around 200 symbols related to clang::WhileStmt. It instantiates TrueMatcher<clang::WhileStmt>, DynCastMatcher<clang::Stmt, , lang::WhileStmt>, BindableMatcher<clang::WhileStmt>, MatcherInterface<clang::WhileStmt>, SingleNodeMatcherInterface<clang::WhileStmt>, VariadicOperatorMatcherInterface<clang::WhileStmt>, Matcher<clang::WhileStmt>::WrappedMatcher, Matcher<clang::WhileStmt>, IdMatcher<clang::WhileStmt>.<u></u><u></u></p>
</div><div><p class="">All of these have virtual methods, which means that there are 2 symbols per constructor, 3 for the destructor and 1 for the vtable. This is without counting the actual methods in the class.<u></u><u></u></p>
</div><div><p class="">We then do more method/class instantiations for the dynamic binding.<u></u><u></u></p></div><div><p class=""><u></u> <u></u></p></div><div><p class="">The main fix I wanted to try is to reduce the number of classes that get instantiated. On the previous fix I removed vector<T> and list<T> for each AST node. I just replaced those uses with plain arrays.<u></u><u></u></p>
</div><div><p class="">I will do another round of fixes. Do you have a target size/sections ?<u></u><u></u></p></div><div><p class=""><u></u> <u></u></p></div><blockquote style="border-style:none none none solid;border-left-color:rgb(204,204,204);border-left-width:1pt;padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<p class="">At the moment, it seems clang cannot be build with visual studio in debug<br>mode; and the large file size of Register.cpp.o and the large number of<br>sections seems to be slowing down the compile process on other platforms<br>
too.<u></u><u></u></p></blockquote><div><p class=""><u></u> <u></u></p></div><div><p class="">In the worst case we could use plan A, which was splitting Register.cpp into many smaller files. We don't need to have all this code in the same compilation unit. I wanted to avoid doing this because it doesn't help to the total size of the objects and makes the code more complex.<u></u><u></u></p>
</div><div><p class=""><u></u> <u></u></p></div><div><p class="">_Sam<u></u><u></u></p></div><div><p class=""> <u></u><u></u></p></div><blockquote style="border-style:none none none solid;border-left-color:rgb(204,204,204);border-left-width:1pt;padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<p class="" style="margin-bottom:12pt"><br>Thanks,<br><br>Kristof<br><br><u></u><u></u></p></blockquote></div><p class=""><u></u> <u></u></p></div></div></div></div></div></div></div></blockquote></div><br></div></div>