[cfe-dev] BUILD_SHARED_LIBS=True breaks some tests

[Chris Bieneman via cfe-dev]

> On Jan 9, 2019, at 10:52 AM, Joel E. Denny <jdenny.ornl at gmail.com> wrote:
> 
> On Tue, Jan 8, 2019 at 6:04 PM Chris Bieneman <chris.bieneman at me.com <mailto:chris.bieneman at me.com>> wrote:
> 
> 
>> On Jan 8, 2019, at 2:44 PM, Joel E. Denny via cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
>> 
>> On Tue, Jan 8, 2019 at 5:14 PM David Greene <dag at cray.com <mailto:dag at cray.com>> wrote:
>> "Joel E. Denny via cfe-dev" <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> writes:
>> 
>> > Do other Clang developers find BUILD_SHARED_LIBS=True useful? Do you
>> > see the above failures? Should there be a bot to test that build
>> > configuration?
>> 
>> I don't use BUILD_SHARED_LIBS=True
>> 
>> One point I'd like to understand is why more people don't use it.  Were you not aware of it, or is it just not beneficial enough to you?  I was very happy to reduce a nearly 4-minute rebuild after a minor change to ~31 sec.  Especially when I have to do that repeatedly, that time difference can determine whether I decide to switch contexts or stay focused.
> 
> How much of that rebuild time is actually link time? I'd guess most.
> 
> Yes.  Even more so because I forgot I have ccache enabled.
>  
> What linker are you using? ld64, lld and gold are all much faster than bfd, so the performance implications may be smaller to other people.
> 
> I just tried LLVM_USE_LINKER=gold and then lld, and lld gave me a better incremental build time, so I'll stick with it.
> 
> The aforementioned ~4 min shrank to ~35s.  However, BUILD_SHARED_LIBS=True shrank that to 3s!
> 
> Continuing to use lld, when I disable ccache (or make a change ccache hasn't seen), it's ~40s and BUILD_SHARED_LIBS=True shrinks it to ~13s.
> 
> The gap is certainly closing, but BUILD_SHARED_LIBS=True still seems appealing.
>  
> Also, using `BUILD_SHARED_LIBS` has a significant impact on execution performance of the final binaries, which impacts test execution speed. So if you aren't struggling with link time, it can be overall better to generate faster loading binaries for the test suite.
> 
> Indeed, ninja check-clang grows from ~10 min to ~20 min when I use BUILD_SHARED_LIBS=True.  Either way, I'd switch contexts instead of waiting, so my development time wouldn't necessarily grow with the latter.
> 
> Nevertheless, whether to use BUILD_SHARED_LIBS=True is definitely a harder choice to make now.
>  
> 
> There are a lot of tradeoffs on performance, but I've strongly advocated that BUILD_SHARED_LIBS should never be used when building distributions for performance reasons, which means it is only really supported as a developer workflow.
> 
> Seems fine to me except that, if it is supported as a developer workflow, the broken tests are a problem.
>  
> 
> Additionally BUILD_SHARED_LIBS is problematic with some of LLVM's design patterns, like cl::opt's global registration, which can deter its usefulness.
> 
> Could you explain a little more, or point to a previous discussion?

cl::opt options are generally globally scoped. When you define one, they are added to a global list of options. No two options can have the same name.

When you link against a shared library, all the globals for that shared library become part of your binary. When you link against a static archive only the globals for any translation units that you need get pulled into your binary.

In the past this has resulted in BUILD_SHARED_LIBS and LLVM_USE_LLVM_DYLIB causing duplicate options to be registered which results in a fatal error (see CommandLine.cpp, addLiteralOption).

-Chris

>  
> 
> Hope this helps explain why it is less widely used than you might have anticipated.
> 
> Definitely.  Thanks!
> 
> Joel
>  
> 
> -Chris
> 
>>  
>> but if that's a supported option, it
>> should be tested.
>> 
>> This argument will, of course, lead to an explosion of builders, as the
>> set of combinations of cmake variables is very large.  It would be
>> useful to gather the various configurations people use and see if there
>> is a set of common-ish combinations that is small enough to regularly
>> test.
>> 
>> We build with RTTI and exceptions enabled and I'll bet there are others
>> out there that do the same.  But AFAIK there are no bots testing those
>> options.
>> 
>> To reduce the configuration space, we could consider combining orthogonal options under a single builder.  That could make debugging some fails a little harder, but it might be worthwhile as it would provide more coverage with less builders.
>> 
>> Thanks.
>> 
>> Joel
>>  
>> 
>>                              -David
>> 
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190109/47879868/attachment.html>