[llvm-dev] moving libfuzzer to compiler-rt?

[Kostya Serebryany via llvm-dev]

On Wed, Jul 12, 2017 at 2:18 PM, George Karpenkov <ekarpenkov at apple.com>
wrote:

>
> On Jul 12, 2017, at 2:16 PM, Kostya Serebryany <kcc at google.com> wrote:
>
>
>
> On Wed, Jul 12, 2017 at 2:11 PM, George Karpenkov via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> > I really like the property of libFuzzer living in its own place so that
>> > it's easy to use without building the world
>>
>> But it’s not: the implementation of the coverage instrumentation
>> is done in one of the sanitizers, so it’s impossible to just use
>> libFuzzer without them.
>>
>
> Mmm. what do you mean?
> libFuzzer implements it's own coverage hooks now.
>
>
> I might be wrong here, but I’m definitely seeing “missing symbol” errors
> every now and then if none
> of the sanitizers was linked (it seems to complain specifically about
> ubsan for some reason).
>

Yea... But those are not coverage


>
>
>
>> Furthermore, I would think that almost all libFuzzer users would use a
>> sanitizer while fuzzing.
>>
>
> Probably.
> Although for equivalence fuzzing (aka differential fuzzing) sanitizers are
> not needed.
>
>
>>
>> I can see a few more benefits of moving to compiler-rt:
>>
>> 1) It would enforce the property
>> that either sanitizers and libfuzzer are both in “runtimes”, or they are
>> both aren’t.
>> Then we can naturally add a dependency from libFuzzer test to sanitizers
>> using CMake.
>> Otherwise, adding a dependency from libfuzzer tests to sanitizers requires
>> hacky if/elses in CMake configuration (and we already have too many of
>> those).
>>
>> 2) Orthogonally to the moving-of-source-code discussion, I would like to
>> move the generated library
>> to the folder where other clang libraries and sanitizers are.
>> That would considerably simplify toolchain installation problem (as we
>> know that sanitizers are already installed
>> correctly in the toolchain) as well as simplifying driver logic for both
>> Clang and Swift.
>> [Diverting even further, I think it would be beneficial to build a
>> dynamic library as well as an archive,
>> and use that at least on Mac, where it can solve the linking issues
>> discussed in https://github.com/google/sanitizers/issues/832.
>> It would be also be much easier if the resulting dylib would be in the
>> same folder where other sanitizers are, due to the fact
>> that we already have the code for setting the expected rpath on the
>> produced binary].
>>
>> This move can of course be done from any source directory, as it’s just a
>> matter of setting a target property.
>> Yet the logic for selecting the folder is a 20-30 lines chunk of CMake,
>> which I would like to avoid copying,
>> and which we could otherwise just reuse from compiler-rt.
>>
>> 3) As Chris Bieneman has previously mentioned in the beginning of this
>> thread,
>> adding libFuzzer to the toolchain installation might be problematic under
>> the current license,
>> which requires binary attribution.
>> Performing the move and resolving the licensing concern would solve that.
>>
>
> Good! :)
>
>
>>
>>
>> > On Jul 12, 2017, at 1:27 PM, Justin Bogner <mail at justinbogner.com>
>> wrote:
>> >
>> > Reid Kleckner via llvm-dev <llvm-dev at lists.llvm.org> writes:
>> >> On Wed, Jul 12, 2017 at 11:30 AM, George Karpenkov via llvm-dev <
>> >> llvm-dev at lists.llvm.org> wrote:
>> >>>
>> >>> I already forgot why we decided not to move the code to compiler-rt.
>> >>>
>> >>> This would solve at least this problem.
>> >>> Since we now have -fsanitize=fuzzer it will actually be pretty
>> natural.
>> >>>
>> >>>
>> >>> Licensing concerns, compiler-rt has a different license.
>> >>>
>> >>> BTW libFuzzer CMake has a crazy amount of hacks to work under Windows,
>> >>> where logic in many parts
>> >>> is entirely different, so any help on testing and fixing arising
>> issues
>> >>> would be much appreciated.
>> >>> All I can do is to make a commit and then try to understand the
>> response
>> >>> from a bots,
>> >>> which is not very efficient.
>> >>>
>> >>
>> >> I got volunteered to help with that, I'll try your patch.
>> >>
>> >> There's more stuff to unpack in the rest of your email, though. I'd
>> really
>> >> rather go to compiler-rt instead of runtimes. runtimes seems like it's
>> >> going the opposite direction from the monorepo efforts.
>> >
>> > I don't see how runtimes is going either for- or against- the monorepo
>> > stuff. We really need a model like runtimes/ rather than a model like
>> > projects/ going forward regardless of where you find the code. This is
>> > necessary if we want to build cross-compiled compiler-rt libraries in
>> > any sort of sane way.
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170712/b60e2d78/attachment.html>