[llvm-dev] Building LLVM's fuzzers

[Justin Bogner via llvm-dev]

Kostya Serebryany <kcc at google.com> writes:
> Interesting.
> This is a relatively new addition (fsanitize-coverage=pc-tables, which is
> now a part of -fsanitize=fuzzer).
> The tests worked (did they? On Mac?) so I thought everything is ok.

It looks like the tests were passing, and I *think* I'd tried my fuzzer
since that change was in place. Maybe something about how we're linking
in compiler-rt makes it more obvious to the linker that these symbols
look dead?

> Yea, we need to make sure the pc-tables are not stripped (this is a
> separate section with globals).
> (I still haven't documented pc-tables, will do soon)
>
> Do you know what's the analog of Wl,-dead_strip on Linux?

I think the closest thing is -Wl,--gc-sections but that might be less
aggressive about it than macOS's linker is.

> On Thu, Aug 24, 2017 at 2:49 PM, Justin Bogner <mail at justinbogner.com>
> wrote:
>
>> George Karpenkov <ekarpenkov at apple.com> writes:
>> > OK so with Kuba’s help I’ve found the error: with optimization, dead
>> > stripping of produced libraries is enabled,
>> > which removes coverage instrumentation.
>> >
>> > However, this has nothing to do with the move to compiler-rt, so I’m
>> > quite skeptical on whether it has worked
>> > beforehand.
>> >
>> > A trivial fix is to do:
>> >
>> > diff --git a/cmake/modules/HandleLLVMOptions.cmake b/cmake/modules/
>> HandleLLVMOptions.cmake
>> > index 04596a6ff63..5465d8d95ba 100644
>> > --- a/cmake/modules/HandleLLVMOptions.cmake
>> > +++ b/cmake/modules/HandleLLVMOptions.cmake
>> > @@ -665,6 +665,9 @@ if(LLVM_USE_SANITIZER)
>> >    endif()
>> >    if (LLVM_USE_SANITIZE_COVERAGE)
>> >      append("-fsanitize=fuzzer-no-link" CMAKE_C_FLAGS CMAKE_CXX_FLAGS)
>> > +
>> > +    # Dead stripping messes up coverage instrumentation.
>> > +    set(LLVM_NO_DEAD_STRIP ON)
>> >    endif()
>> >  endif()
>> >
>> > Any arguments against that?
>>
>> We shouldn't do this. We really only want to prevent dead stripping of
>> the counters themselves - disabling it completely isn't very nice.
>>
>> > Apparently, a better way is to follow ASAN instrumentation pass,
>> > which uses some magic to protect against dead-stripping.
>>
>> I thought this was already being done - how else did it work before?
>>
>> >> On Aug 24, 2017, at 11:29 AM, Justin Bogner <mail at justinbogner.com>
>> wrote:
>> >>
>> >> (kcc, george: sorry for the re-send, the first was from a non-list email
>> >> address)
>> >>
>> >> My configuration for building the fuzzers in the LLVM tree doesn't seem
>> to
>> >> work any more (possibly as of moving libFuzzer to compiler-rt, but there
>> >> have been a few other changes in the last week or so that may be
>> related).
>> >>
>> >> I'm building with a fresh top-of-tree clang and setting
>> >> -DLLVM_USE_SANITIZER=Address and -DLLVM_USE_SANITIZE_COVERAGE=On, which
>> >> was working before:
>> >>
>> >>  % cmake -GNinja \
>> >>          -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=On \
>> >>          -DLLVM_ENABLE_WERROR=On \
>> >>          -DLLVM_USE_SANITIZER=Address -DLLVM_USE_SANITIZE_COVERAGE=On \
>> >>          -DCMAKE_C_COMPILER=$HOME/llvm-lkgc/bin/clang \
>> >>          $HOME/code/llvm-src
>> >>
>> >> But when I run any of the fuzzers, it looks like the sanitizer coverage
>> >> hasn't been set up correctly:
>> >>
>> >>  % ./bin/llvm-as-fuzzer
>>                                    2017-08-24 11:14:33
>> >>  INFO: Seed: 4089166883
>> >>  INFO: Loaded 1 modules   (50607 guards): 50607 [0x10e14ef80,
>> 0x10e18063c),
>> >>  INFO: Loaded 1 PC tables (0 PCs): 0 [0x10e2870a8,0x10e2870a8),
>> >>  ERROR: The size of coverage PC tables does not match the number of
>> instrumented PCs. This might be a bug in the compiler, please contact the
>> libFuzzer developers.
>> >>
>> >> From the build logs, it looks like we're now building objects with these
>> >> sanitizer flags:
>> >>
>> >>  -fsanitize=address
>> >>  -fsanitize-address-use-after-scope
>> >>  -fsanitize=fuzzer-no-link
>> >>
>> >> We're then linking the fuzzer binaries with these:
>> >>
>> >>  -fsanitize=address
>> >>  -fsanitize-address-use-after-scope
>> >>  -fsanitize=fuzzer-no-link
>> >>  -fsanitize=fuzzer
>> >>
>> >> Any idea what's wrong or where to start looking?
>>