[cfe-dev] [llvm-dev] Is it really valid to discard externally instantiated functions from a TU when marked inline?

[Louis Dionne via cfe-dev]

> On Jul 20, 2018, at 15:35, Petr Hosek <phosek at chromium.org> wrote:
> 
> I might be completely wrong about this, but I don't think libFuzzer relies on __always_inline, at least not intentionally. The way libFuzzer statically links libc++ is more complicated than simply merging all object files into a single archive. What we do is first compiling libc++ with _LIBCPP_ABI_VERSION=Fuzzer so all symbols that would normally end up in std::__1 (or __2 if you use ABI v2) namespace end up in std::__Fuzzer namespace (this may not even be needed given what follows). We also also disable the default visibility annotations by setting _LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS and compile libc++ with -fvisibility=hidden set so all symbols should end up as hidden. We then link all libFuzzer and libc++ object files into a single relocatable .o file and we use objcopy --localize-hidden to make all symbols that were marked as hidden (i.e. all libc++ symbols) local. Given that, there should be no collisions between libFuzzer’s internal libc++ copy and libc++ application is using (even when linking libc++ statically), so I don't think removing __always_inline would make any difference for this use case.

If that’s how it works, then indeed I don’t think libFuzzer is a problem. The part that saves us is the one where hidden symbols are marked as local. In that case, it may be worthwhile to consider a complete redesign of libc++’s approach to visibility, which could yield significant simplifications. I’ll think about that once we’ve solved the immediate problem of `__always_inline__`.

Louis

> 
> On Fri, Jul 20, 2018 at 8:35 AM Louis Dionne via cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
> 
>> On Jul 19, 2018, at 19:51, Dean Michael Berris <dean.berris at gmail.com <mailto:dean.berris at gmail.com>> wrote:
>> 
>> 
>> 
>>> On 20 Jul 2018, at 01:49, James Y Knight via cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
>>> 
>>> 
>>> I haven't looked at what libfuzzer does, but as you've described it, I'd say this _doesn't_ seem a reasonable use-case. If you want to link libc++ (or any other library!) hidden inside your library, you should be using a version-script to only expose symbols that you intend to expose.
>>> 
>> 
>> In XRay, we go out of our way to not use any standard-library defined symbols that need linkage. We have a tests which ensure that we are able to link the XRay runtime without the standard library dependency. This has been discussed before, I think, precisely in this context.
>> 
>> The options for libFuzzer might be more limited since it’s a special use-case, and I’m sure unless the fuzzer runtime removes dependencies on standard library symbols that libc++ shouldn’t be prioritising this use-case. Another way of saying this is that the cost of ensuring that libFuzzer continues to work despite changes to libc++ should be borne by libFuzzer, not the other way around.
> 
> Well, that’s also my impression, but I don’t know how we can actually change this given there are clients using it and the status quo usually wins when no consensus can be reached. What I suggest is that we instead start implementing my proposal at http://lists.llvm.org/pipermail/cfe-dev/2018-July/058457.html <http://lists.llvm.org/pipermail/cfe-dev/2018-July/058457.html> (I’m waiting for reviews on https://reviews.llvm.org/D49240 <https://reviews.llvm.org/D49240> BTW), which will solve most concrete problems we have right now. In the not too distant future, we can then evaluate the feasibility of removing support for interoperability of TUs built with different versions and the benefits of doing so — as a separate effort (which I’d be glad to volunteer for).
> 
> My primary concern at this point is about moving forward and solving the concrete problems we have with the use of __always_inline__, and I’m willing to compromise to get there, and to do it in stages if I have to.
> 
> Cheers,
> Louis
> 
> 
>> I don’t know how much work is required to use a work-around as James describes with symbol visibility or “privatising” the symbols libFuzzer uses from libc++. It could be less than trying to remove the dependency on libc++ instead.
>> 
>> -- Dean
>> 
> 
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180723/c6698402/attachment.html>