[cfe-dev] Big program size increase with LibC++ v3.7 versus v3.6

Tue Oct 13 01:47:56 PDT 2015

Hmm, I have not explicitly or intentionally done so.  For the most part I just build the library and run its test-suite to verify the compiler which is my principal focus.  But this means that I have not dug into the library implementations to find out how they work, I just assume that they do.  I didn't see anything in the configuration that controls this, is there an additional define that I need to specify or ensure is not specified?

The main things I have to do is use static libraries because there is no dynamic runtime environment for shared libraries.  And I also suppress exception handling and threads because there is no support for them on the platform either.  But I am not aware of having disabled the use of extern templates and explicit instantiation.

Is there some option of source files I should look at first to see if this has happened?

My build-system does not use the normal process, but it mimics what it should do.  The main reason for this is that I could not see how to use the provided build system to build with the clang cross-compiler I have just build using.  I know that there are changes happening to support this scenario under CMake, but it doesn't seem to work yet in v3.7 and attempts to build with the host compiler.

Thanks for your help,

	MartinO

-----Original Message-----
From: dexonsmith at apple.com [mailto:dexonsmith at apple.com] 
Sent: 12 October 2015 22:37
To: Martin J. O'Riordan <martin.oriordan at movidius.com>
Cc: Marshall Clow <mclow.lists at gmail.com>; Eric Fiselier <eric at efcs.ca>; Evgenii Stepanov <eugeni.stepanov at gmail.com>; cfe-dev at lists.llvm.org; Justin Bogner <mail at justinbogner.com>
Subject: Re: [cfe-dev] Big program size increase with LibC++ v3.7 versus v3.6

I can't imagine it's directly related, but somehow this reminds me strongly of r215740:
--
    Revert "Turn off extern templates for most uses."

    Turning off explicit template instantiation leads to a pretty
    significant build time and code size cost. We're better off dealing
    with ABI incompatibility issues that come up in a less heavy handed
    way.

    This reverts commit r189610.
--

Maybe somehow your implementation has disabled explicit template instantiation in the headers?

> On 2015-Oct-12, at 13:30, Martin J. O'Riordan via cfe-dev <cfe-dev at lists.llvm.org> wrote:
> 
> Oops ‘Eric’ I auto-piloted my niece’s name!!  Sorry about that!
>  
> From: Martin J. O'Riordan [mailto:martin.oriordan at movidius.com]
> Sent: 12 October 2015 21:29
> To: 'Marshall Clow' <mclow.lists at gmail.com>; 'Eric Fiselier' 
> <eric at efcs.ca>; 'Evgenii Stepanov' <eugeni.stepanov at gmail.com>; 
> 'cfe-dev at lists.llvm.org' <cfe-dev at lists.llvm.org>
> Subject: RE: [cfe-dev] Big program size increase with LibC++ v3.7 
> versus v3.6
>  
> Thanks Erica and Marshall for trying this out and giving me some data points.  The changes to LibC++ look incremental, but whatever is going on it is having a catastrophic impact on my implementation.  I have a suspicion that our lack of support for weak extern linkage may be a factor.
>  
>             MartinO
>  
> From: Marshall Clow [mailto:mclow.lists at gmail.com]
> Sent: 12 October 2015 17:36
> To: Eric Fiselier <eric at efcs.ca>; Martin J. O'Riordan 
> <martin.oriordan at movidius.com>; Marshall Clow <mclow.lists at gmail.com>; 
> Evgenii Stepanov <eugeni.stepanov at gmail.com>; cfe-dev at lists.llvm.org
> Subject: Re: [cfe-dev] Big program size increase with LibC++ v3.7 
> versus v3.6
>  
> On Sun, Oct 11, 2015 at 3:36 PM, Eric Fiselier <eric at efcs.ca> wrote:
>> I tried building and testing libc++ 3.6 on Linux but I couldn't easily reproduce. I built the 3.6 sources (using the ToT buildsystem) with -DCMAKE_BUILD_TYPE=RELEASE -DLIBCXX_ENABLE_SHARED=OFF -DLIBCXX_ENABLE_STATIC_ABI_LIBRARY=ON and then built the egrep test exactly like the test-suite normally would. I did the same with the current ToT. 
>>  
>> I did see a 10% size of the egrep test but nothing like 2x or 3x.
>  
> That's what I'm seeing on Mac OS X as well
>  
> 98K for 3.6, 106K for 3.7.  (both built with -O3) 320K for both 3.6 
> and 3.7 when built without -O3
>  
> -- Marshall
>  
>  
>> On Sun, Oct 11, 2015 at 3:40 PM, Eric Fiselier <eric at efcs.ca> wrote:
>>> Off the top of my head I can't think of any one change that would cause this. CCing Marshall Clow to see if he knows. I'm also CC'ing Evgenii because he knows a lot about the visibility and inlining of libc++'s symbols.
>>>  
>>> Originally I thought it might be related to the external 
>>> instantiations of std::basic_string, but nothing should have changed between 3.6 and 3.7 in that regard.
>>>  
>>> > Are there any ways of building the library with minimal or no locale support?
>>>  
>>> No and I don't think libc++ should add one. We already support way 
>>> to much feature sub-setting.
>>>  
>>>  
>>>  
>>> /Eric
>>>  
>>> On Sun, Oct 11, 2015 at 6:34 AM, Martin J. O'Riordan via cfe-dev <cfe-dev at lists.llvm.org> wrote:
>>>> Hi CFE Devs,
>>>> 
>>>> I have recently completed upgrading our CLang/LLVM based compiler 
>>>> from v3.6 to v3.7, but I was noticing some significant regressions 
>>>> in the LibC++ test-suite.  Something has changed that is resulting 
>>>> in the code-size being about 3X larger and the data size being about 2.7X larger.
>>>> 
>>>> Initially I suspected the compiler was at fault, so I did a series 
>>>> of builds and comparisons to narrow down where the problem changes 
>>>> occurred.  I'll take one test as an example:
>>>> 
>>>>    projects/libcxx/test/std/re/re.alg/re.alg.search/egrep.pass.cpp
>>>> 
>>>> With the v3.6 compiler and the v3.6 LibC++ library, this test was 
>>>> resulting in a program with 122,916 Bytes of code and 6,240 Bytes 
>>>> of data.  With the
>>>> v3.7 compiler and the v3.7 LibC++ this became 366,708 and 16,680 
>>>> respectively!  I don't know how this compares with other targets, 
>>>> so I can only discuss our SHAVE program size.
>>>> 
>>>> At first I thought that perhaps we had broken something in LLVM, 
>>>> inlining for example, so I tried the following:
>>>> 
>>>>    Use the v3.7 compiler to run the tests, but with the v3.6 built LibC++
>>>>    library and v3.6 LibC++ headers.  This brought the figures to 127,940
>>>>    and 6,240 respectively; much closer to the original.
>>>> 
>>>> But this still didn't rule out a fault in the compiler.  So I tried 
>>>> the
>>>> following:
>>>> 
>>>>    Use the v3.7 compiler to build the v3.6 LibC++ library, and again run
>>>>    the tests using this library and the v3.6 LibC++ headers.  This time I
>>>>    got 127,940 and 6,288 Bytes respectively; very close to the v3.
>>>>    compiler figures which would indicate that the compiler itself has not
>>>>    caused this regression.
>>>> 
>>>> So I am wondering what has happened in the sources for LibC++ v3.7 
>>>> that could cause this?
>>>> 
>>>> What appears to be happening, is that the library is pulling in 
>>>> many more symbols (hundreds) from the libraries even though they 
>>>> are never actually executed, and a lot of these are related to 
>>>> 'char_traits' and wide-characters; especially in the streams and 
>>>> stream iterators.  I haven't previously delved into the sources for 
>>>> the LibC++ library as my primary focus is on the compiler (backend 
>>>> mainly), so I don't have enough experience of the implementation of 
>>>> LibC++ to determine why this is.  Examining the header changes 
>>>> doesn't reveal any obvious smoking gun, though I did notice that there are some significant 'traits' related changes to '<streambuf>'.
>>>> 
>>>> Our platform is for embedded deployment, so I we don't need rich 
>>>> locale support (C locale is fine), nor Unicode or wide-characters.  
>>>> But I don't see any configuration options in the sources for LibC++ 
>>>> that allows these to be tuned for embedded systems.  Are there any 
>>>> ways of building the library with minimal or no locale support?
>>>> 
>>>> We build LibC++ as a static library with RTTI enabled, but with 
>>>> threads and exception handling disabled ('__SINGLE_THREAD__', 
>>>> '_LIBCPP_NO_EXCEPTIONS', '_LIBCPP_BUILD_STATIC', 
>>>> '_LIBCPP_HAS_NO_THREADS' and '_LIBCPP_HAS_NO_MONOTONIC_CLOCK' are all defined).
>>>> 
>>>> For LibC we are using Newlib v2.2.0, and our assembler does not 
>>>> support weak externals which might be relevant, but we do support 
>>>> ODR linkage and garbage collection in the linker (all data & functions in discrete sections).
>>>> 
>>>> Is anybody else experiencing this kind of size increase in C++ 
>>>> programs since migrating to v3.7?  I have 507 LibC++ v3.7 
>>>> test-cases which have similarly increased versus the v3.6 version, 
>>>> mostly in the area of iterators and streams.
>>>> 
>>>> Thanks,
>>>> 
>>>>         MartinO - Movidius Ltd.
>>>> 
>>>> 
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>  
>>  
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev