[cfe-dev] 'too many sections' when building an object file for llvm/clang

[Samuel Benzaquen]

On Fri, Jun 7, 2013 at 2:22 PM, Manish Verma <manish.verma at arm.com> wrote:

> Hi,
>

Hi,
I am the person working on the dynamic layer (ASTMatchers/Dynamic/) on top
the matcher library.


>
> I have been discussing the same issue on llvm-commit mailing-list.
>
> http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20130603/177054.
> html
>
> At the moment, the debug build of LLVM/Clang is broken on Windows.
> This is because when compiling ASTMatchers/Dynamic/Registry.cpp,
> the compilers hit the allowed limit of 2^16 sections in the
> COFF file.
>

Looking at the sections, it seems that we have way too many functions
defined in the compilation unit.
The matchers in ASTMatchers.h rely a lot on templates and macros and most
of the matcher code is directly on the ASTMatchers.h file.
Each matcher instantiation seem to generate a couple hundred.
The dynamic registry is instantiating all* the matchers in Registry.cpp. I
think only the test file (ASTMatchersTest.cpp) would instantiate that many
matchers together in one compilation unit before.

I see a few solutions for this:
 1. For any non-template matcher, move its code to an ASTMatchers.cpp to be
created. This would take away a big chunk of the size of Registry.cpp. I
don't know if this will just move the problem to ASTMatchers.cpp instead.
 2. Split Registry.cpp in smaller compilation units, each with a subset of
the matchers. It should be easier than (1).

* the goal is to have them all, but we are not there yet, so the problem
would get worse later.

For MSVC it is possible to work around this problem by compiling
> with /bigobj command line option. However, I don't know of an
> equivalent command line option for Min-GW gcc.
>
> On Linux, Registry.o (debug-mode) is about 34M in size and has
> 36986 sections. I can expect it impacting the link-time and memory
> usage.
>

This library is separate from ASTMatchers on purpose to prevent bloat to
anyone not explicitly requiring the dynamic implementation.
If this is being built/linked for clang/clang++ then it is a mistake.


>
> The last commit has resulted in the build failure, but I don't think
> it is responsible for the failure.
>

On the last commit (or one of the latest) I expanded the list of matchers
to more than 150. I expect this is what caused it to go over the limit.

_Sam


> I don't know what should be the correct solution. Hence, I am raising
> it here for discussion.


> Regards,
> Manish
>
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130607/69ea3330/attachment.html>