[cfe-dev] 'too many sections' again when building ASTMatchers/Dynamic/Registry.cpp on Windows

[Samuel Benzaquen]

On Fri, Aug 30, 2013 at 3:40 AM, Kristof Beyls <kristof.beyls at arm.com>wrote:

> Hi Samuel,****
>
> ** **
>
> Thanks for the quick response.****
>
> I agree that the best way forward would be to reduce the number of
> sections/symbols produced
> in Register.cpp.o. I’m afraid I don’t know enough about this part of clang
> to know how feasible
> that is.****
>
> I’m afraid I don’t really have a good target for the maximum number of
> sections to be produced
> in a single object file. I think the target should be to at least halve
> the number of sections of what
> is currently produced, and hope that future changes will not be creeping
> up the number of
> sections again.
>

Just as fyi, I already submitted one change that took out ~10% of the
symbols.
I have 2 more for review with similar benefit (
http://llvm-reviews.chandlerc.com/D1564 and
http://llvm-reviews.chandlerc.com/D1560).
I think there are a couple easy ones after those, but I haven't explored
them yet.
_Sam

> ****
>
> ** **
>
> Thanks,****
>
> ** **
>
> Kristof****
>
> ** **
>
> *From:* Samuel Benzaquen [mailto:sbenza at google.com]
> *Sent:* 29 August 2013 23:09
> *To:* Kristof Beyls
> *Cc:* Clang Dev; Nicholas Braden; Manuel Klimek
> *Subject:* Re: 'too many sections' again when building
> ASTMatchers/Dynamic/Registry.cpp on Windows****
>
> ** **
>
> On Thu, Aug 29, 2013 at 3:49 PM, Kristof Beyls <kristof.beyls at arm.com>
> wrote:****
>
> Hi Samuel,
>
> We've observed that in the past few days, debug-builds with Visual Studio
> have started failing again because too many sections are generated for
> ASTMatchers/Dynamic/Registry.cpp.
>
> This has been observed and discussed before in this email thread:
> http://lists.cs.uiuc.edu/pipermail/cfe-dev/2013-June/030102.html.
>
> It seems that since then, bit by bit, the number of sections for
> Registry.cpp
> has been creeping up. I've done a quick experiment on linux to measure how
> the size and the number of sections in Registry.cpp.o has been increasing
> over
> time. The table does show that your fix in r183768 did reduce the number of
> sections a lot; but since then, the number of sections has been increasing
> steadily and quite rapidly.
>
>                 Size of Register.cpp.o
> svn id      bytes               sections
> 189448  27122264        28108
> 189353  27204656        28259
> 189031  26473624        27284
> 187500  26451288        26952
> 186000  21940824        23119
> 183769  19289872        20544
> 183768  33647984        35738
>
> You did say on the previous email thread that you had a few more ideas on
> how
> to reduce the number of sections in Register.cpp.o. I'm wondering if you
> would
> be able to share the ideas you've got?****
>
> ** **
>
> The problem is that there are a lot of AST nodes, and each node is getting
> a whole bunch of classes instantiated, many of them with virtual methods.*
> ***
>
> For example, there are around 200 symbols related to clang::WhileStmt. It
> instantiates TrueMatcher<clang::WhileStmt>, DynCastMatcher<clang::Stmt, ,
> lang::WhileStmt>, BindableMatcher<clang::WhileStmt>,
> MatcherInterface<clang::WhileStmt>,
> SingleNodeMatcherInterface<clang::WhileStmt>,
> VariadicOperatorMatcherInterface<clang::WhileStmt>,
> Matcher<clang::WhileStmt>::WrappedMatcher, Matcher<clang::WhileStmt>,
> IdMatcher<clang::WhileStmt>.****
>
> All of these have virtual methods, which means that there are 2 symbols
> per constructor, 3 for the destructor and 1 for the vtable. This is without
> counting the actual methods in the class.****
>
> We then do more method/class instantiations for the dynamic binding.****
>
> ** **
>
> The main fix I wanted to try is to reduce the number of classes that get
> instantiated. On the previous fix I removed vector<T> and list<T> for each
> AST node. I just replaced those uses with plain arrays.****
>
> I will do another round of fixes. Do you have a target size/sections ?****
>
> ** **
>
> At the moment, it seems clang cannot be build with visual studio in debug
> mode; and the large file size of Register.cpp.o and the large number of
> sections seems to be slowing down the compile process on other platforms
> too.****
>
> ** **
>
> In the worst case we could use plan A, which was splitting Register.cpp
> into many smaller files. We don't need to have all this code in the same
> compilation unit. I wanted to avoid doing this because it doesn't help to
> the total size of the objects and makes the code more complex.****
>
> ** **
>
> _Sam****
>
>  ****
>
>
> Thanks,
>
> Kristof
>
> ****
>
> ** **
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130830/cd4fdc5e/attachment.html>