[llvm-dev] Building LLVM's fuzzers

[Saleem Abdulrasool via llvm-dev]

It can be made as aggressive.  You need to compile with
`-ffunction-sections` `-fdata-sections` and then link with `-Xlinker
--gc-sections`.  ld64 is more aggressive due to `.subsections_via_symbols`
(aka the two compile flags).

On Thu, Aug 24, 2017 at 3:10 PM, Justin Bogner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Kostya Serebryany <kcc at google.com> writes:
> > Interesting.
> > This is a relatively new addition (fsanitize-coverage=pc-tables, which is
> > now a part of -fsanitize=fuzzer).
> > The tests worked (did they? On Mac?) so I thought everything is ok.
>
> It looks like the tests were passing, and I *think* I'd tried my fuzzer
> since that change was in place. Maybe something about how we're linking
> in compiler-rt makes it more obvious to the linker that these symbols
> look dead?
>
> > Yea, we need to make sure the pc-tables are not stripped (this is a
> > separate section with globals).
> > (I still haven't documented pc-tables, will do soon)
> >
> > Do you know what's the analog of Wl,-dead_strip on Linux?
>
> I think the closest thing is -Wl,--gc-sections but that might be less
> aggressive about it than macOS's linker is.
>
> > On Thu, Aug 24, 2017 at 2:49 PM, Justin Bogner <mail at justinbogner.com>
> > wrote:
> >
> >> George Karpenkov <ekarpenkov at apple.com> writes:
> >> > OK so with Kuba’s help I’ve found the error: with optimization, dead
> >> > stripping of produced libraries is enabled,
> >> > which removes coverage instrumentation.
> >> >
> >> > However, this has nothing to do with the move to compiler-rt, so I’m
> >> > quite skeptical on whether it has worked
> >> > beforehand.
> >> >
> >> > A trivial fix is to do:
> >> >
> >> > diff --git a/cmake/modules/HandleLLVMOptions.cmake b/cmake/modules/
> >> HandleLLVMOptions.cmake
> >> > index 04596a6ff63..5465d8d95ba 100644
> >> > --- a/cmake/modules/HandleLLVMOptions.cmake
> >> > +++ b/cmake/modules/HandleLLVMOptions.cmake
> >> > @@ -665,6 +665,9 @@ if(LLVM_USE_SANITIZER)
> >> >    endif()
> >> >    if (LLVM_USE_SANITIZE_COVERAGE)
> >> >      append("-fsanitize=fuzzer-no-link" CMAKE_C_FLAGS
> CMAKE_CXX_FLAGS)
> >> > +
> >> > +    # Dead stripping messes up coverage instrumentation.
> >> > +    set(LLVM_NO_DEAD_STRIP ON)
> >> >    endif()
> >> >  endif()
> >> >
> >> > Any arguments against that?
> >>
> >> We shouldn't do this. We really only want to prevent dead stripping of
> >> the counters themselves - disabling it completely isn't very nice.
> >>
> >> > Apparently, a better way is to follow ASAN instrumentation pass,
> >> > which uses some magic to protect against dead-stripping.
> >>
> >> I thought this was already being done - how else did it work before?
> >>
> >> >> On Aug 24, 2017, at 11:29 AM, Justin Bogner <mail at justinbogner.com>
> >> wrote:
> >> >>
> >> >> (kcc, george: sorry for the re-send, the first was from a non-list
> email
> >> >> address)
> >> >>
> >> >> My configuration for building the fuzzers in the LLVM tree doesn't
> seem
> >> to
> >> >> work any more (possibly as of moving libFuzzer to compiler-rt, but
> there
> >> >> have been a few other changes in the last week or so that may be
> >> related).
> >> >>
> >> >> I'm building with a fresh top-of-tree clang and setting
> >> >> -DLLVM_USE_SANITIZER=Address and -DLLVM_USE_SANITIZE_COVERAGE=On,
> which
> >> >> was working before:
> >> >>
> >> >>  % cmake -GNinja \
> >> >>          -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=On \
> >> >>          -DLLVM_ENABLE_WERROR=On \
> >> >>          -DLLVM_USE_SANITIZER=Address -DLLVM_USE_SANITIZE_COVERAGE=On
> \
> >> >>          -DCMAKE_C_COMPILER=$HOME/llvm-lkgc/bin/clang \
> >> >>          $HOME/code/llvm-src
> >> >>
> >> >> But when I run any of the fuzzers, it looks like the sanitizer
> coverage
> >> >> hasn't been set up correctly:
> >> >>
> >> >>  % ./bin/llvm-as-fuzzer
> >>                                    2017-08-24 11:14:33
> >> >>  INFO: Seed: 4089166883
> >> >>  INFO: Loaded 1 modules   (50607 guards): 50607 [0x10e14ef80,
> >> 0x10e18063c),
> >> >>  INFO: Loaded 1 PC tables (0 PCs): 0 [0x10e2870a8,0x10e2870a8),
> >> >>  ERROR: The size of coverage PC tables does not match the number of
> >> instrumented PCs. This might be a bug in the compiler, please contact
> the
> >> libFuzzer developers.
> >> >>
> >> >> From the build logs, it looks like we're now building objects with
> these
> >> >> sanitizer flags:
> >> >>
> >> >>  -fsanitize=address
> >> >>  -fsanitize-address-use-after-scope
> >> >>  -fsanitize=fuzzer-no-link
> >> >>
> >> >> We're then linking the fuzzer binaries with these:
> >> >>
> >> >>  -fsanitize=address
> >> >>  -fsanitize-address-use-after-scope
> >> >>  -fsanitize=fuzzer-no-link
> >> >>  -fsanitize=fuzzer
> >> >>
> >> >> Any idea what's wrong or where to start looking?
> >>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>



-- 
Saleem Abdulrasool
compnerd (at) compnerd (dot) org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170916/44fd8463/attachment.html>