[llvm-dev] moving libfuzzer to compiler-rt?

Wed Jul 12 14:18:14 PDT 2017

> On Jul 12, 2017, at 2:16 PM, Kostya Serebryany <kcc at google.com> wrote:
> 
> 
> 
> On Wed, Jul 12, 2017 at 2:11 PM, George Karpenkov via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> > I really like the property of libFuzzer living in its own place so that
> > it's easy to use without building the world
> 
> But it’s not: the implementation of the coverage instrumentation
> is done in one of the sanitizers, so it’s impossible to just use libFuzzer without them.
> 
> Mmm. what do you mean? 
> libFuzzer implements it's own coverage hooks now. 

I might be wrong here, but I’m definitely seeing “missing symbol” errors every now and then if none
of the sanitizers was linked (it seems to complain specifically about ubsan for some reason).

>  
> Furthermore, I would think that almost all libFuzzer users would use a sanitizer while fuzzing.
> 
> Probably. 
> Although for equivalence fuzzing (aka differential fuzzing) sanitizers are not needed.  
>  
> 
> I can see a few more benefits of moving to compiler-rt:
> 
> 1) It would enforce the property
> that either sanitizers and libfuzzer are both in “runtimes”, or they are both aren’t.
> Then we can naturally add a dependency from libFuzzer test to sanitizers using CMake.
> Otherwise, adding a dependency from libfuzzer tests to sanitizers requires
> hacky if/elses in CMake configuration (and we already have too many of those).
> 
> 2) Orthogonally to the moving-of-source-code discussion, I would like to move the generated library
> to the folder where other clang libraries and sanitizers are.
> That would considerably simplify toolchain installation problem (as we know that sanitizers are already installed
> correctly in the toolchain) as well as simplifying driver logic for both Clang and Swift.
> [Diverting even further, I think it would be beneficial to build a dynamic library as well as an archive,
> and use that at least on Mac, where it can solve the linking issues discussed in https://github.com/google/sanitizers/issues/832 <https://github.com/google/sanitizers/issues/832>.
> It would be also be much easier if the resulting dylib would be in the same folder where other sanitizers are, due to the fact
> that we already have the code for setting the expected rpath on the produced binary].
> 
> This move can of course be done from any source directory, as it’s just a matter of setting a target property.
> Yet the logic for selecting the folder is a 20-30 lines chunk of CMake, which I would like to avoid copying,
> and which we could otherwise just reuse from compiler-rt.
> 
> 3) As Chris Bieneman has previously mentioned in the beginning of this thread,
> adding libFuzzer to the toolchain installation might be problematic under the current license,
> which requires binary attribution.
> Performing the move and resolving the licensing concern would solve that.
> 
> Good! :) 
>  
> 
> 
> > On Jul 12, 2017, at 1:27 PM, Justin Bogner <mail at justinbogner.com <mailto:mail at justinbogner.com>> wrote:
> >
> > Reid Kleckner via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> writes:
> >> On Wed, Jul 12, 2017 at 11:30 AM, George Karpenkov via llvm-dev <
> >> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> >>>
> >>> I already forgot why we decided not to move the code to compiler-rt.
> >>>
> >>> This would solve at least this problem.
> >>> Since we now have -fsanitize=fuzzer it will actually be pretty natural.
> >>>
> >>>
> >>> Licensing concerns, compiler-rt has a different license.
> >>>
> >>> BTW libFuzzer CMake has a crazy amount of hacks to work under Windows,
> >>> where logic in many parts
> >>> is entirely different, so any help on testing and fixing arising issues
> >>> would be much appreciated.
> >>> All I can do is to make a commit and then try to understand the response
> >>> from a bots,
> >>> which is not very efficient.
> >>>
> >>
> >> I got volunteered to help with that, I'll try your patch.
> >>
> >> There's more stuff to unpack in the rest of your email, though. I'd really
> >> rather go to compiler-rt instead of runtimes. runtimes seems like it's
> >> going the opposite direction from the monorepo efforts.
> >
> > I don't see how runtimes is going either for- or against- the monorepo
> > stuff. We really need a model like runtimes/ rather than a model like
> > projects/ going forward regardless of where you find the code. This is
> > necessary if we want to build cross-compiled compiler-rt libraries in
> > any sort of sane way.
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170712/8851ae7c/attachment.html>