[llvm-dev] Building LLVM's fuzzers

[George Karpenkov via llvm-dev]

> On Aug 24, 2017, at 3:10 PM, Justin Bogner <mail at justinbogner.com> wrote:
> 
> Kostya Serebryany <kcc at google.com> writes:
>> Interesting.
>> This is a relatively new addition (fsanitize-coverage=pc-tables, which is
>> now a part of -fsanitize=fuzzer).
>> The tests worked (did they? On Mac?) so I thought everything is ok.
> 
> It looks like the tests were passing, and I *think* I'd tried my fuzzer
> since that change was in place.

that wouldn’t have helped: it was specifying coverage flags explicitly,
which did not include the new flag in question.

> Maybe something about how we're linking
> in compiler-rt makes it more obvious to the linker that these symbols
> look dead?
> 
>> Yea, we need to make sure the pc-tables are not stripped (this is a
>> separate section with globals).
>> (I still haven't documented pc-tables, will do soon)
>> 
>> Do you know what's the analog of Wl,-dead_strip on Linux?
> 
> I think the closest thing is -Wl,--gc-sections but that might be less
> aggressive about it than macOS's linker is.
> 
>> On Thu, Aug 24, 2017 at 2:49 PM, Justin Bogner <mail at justinbogner.com>
>> wrote:
>> 
>>> George Karpenkov <ekarpenkov at apple.com> writes:
>>>> OK so with Kuba’s help I’ve found the error: with optimization, dead
>>>> stripping of produced libraries is enabled,
>>>> which removes coverage instrumentation.
>>>> 
>>>> However, this has nothing to do with the move to compiler-rt, so I’m
>>>> quite skeptical on whether it has worked
>>>> beforehand.
>>>> 
>>>> A trivial fix is to do:
>>>> 
>>>> diff --git a/cmake/modules/HandleLLVMOptions.cmake b/cmake/modules/
>>> HandleLLVMOptions.cmake
>>>> index 04596a6ff63..5465d8d95ba 100644
>>>> --- a/cmake/modules/HandleLLVMOptions.cmake
>>>> +++ b/cmake/modules/HandleLLVMOptions.cmake
>>>> @@ -665,6 +665,9 @@ if(LLVM_USE_SANITIZER)
>>>>   endif()
>>>>   if (LLVM_USE_SANITIZE_COVERAGE)
>>>>     append("-fsanitize=fuzzer-no-link" CMAKE_C_FLAGS CMAKE_CXX_FLAGS)
>>>> +
>>>> +    # Dead stripping messes up coverage instrumentation.
>>>> +    set(LLVM_NO_DEAD_STRIP ON)
>>>>   endif()
>>>> endif()
>>>> 
>>>> Any arguments against that?
>>> 
>>> We shouldn't do this. We really only want to prevent dead stripping of
>>> the counters themselves - disabling it completely isn't very nice.
>>> 
>>>> Apparently, a better way is to follow ASAN instrumentation pass,
>>>> which uses some magic to protect against dead-stripping.
>>> 
>>> I thought this was already being done - how else did it work before?
>>> 
>>>>> On Aug 24, 2017, at 11:29 AM, Justin Bogner <mail at justinbogner.com>
>>> wrote:
>>>>> 
>>>>> (kcc, george: sorry for the re-send, the first was from a non-list email
>>>>> address)
>>>>> 
>>>>> My configuration for building the fuzzers in the LLVM tree doesn't seem
>>> to
>>>>> work any more (possibly as of moving libFuzzer to compiler-rt, but there
>>>>> have been a few other changes in the last week or so that may be
>>> related).
>>>>> 
>>>>> I'm building with a fresh top-of-tree clang and setting
>>>>> -DLLVM_USE_SANITIZER=Address and -DLLVM_USE_SANITIZE_COVERAGE=On, which
>>>>> was working before:
>>>>> 
>>>>> % cmake -GNinja \
>>>>>         -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=On \
>>>>>         -DLLVM_ENABLE_WERROR=On \
>>>>>         -DLLVM_USE_SANITIZER=Address -DLLVM_USE_SANITIZE_COVERAGE=On \
>>>>>         -DCMAKE_C_COMPILER=$HOME/llvm-lkgc/bin/clang \
>>>>>         $HOME/code/llvm-src
>>>>> 
>>>>> But when I run any of the fuzzers, it looks like the sanitizer coverage
>>>>> hasn't been set up correctly:
>>>>> 
>>>>> % ./bin/llvm-as-fuzzer
>>>                                   2017-08-24 11:14:33
>>>>> INFO: Seed: 4089166883
>>>>> INFO: Loaded 1 modules   (50607 guards): 50607 [0x10e14ef80,
>>> 0x10e18063c),
>>>>> INFO: Loaded 1 PC tables (0 PCs): 0 [0x10e2870a8,0x10e2870a8),
>>>>> ERROR: The size of coverage PC tables does not match the number of
>>> instrumented PCs. This might be a bug in the compiler, please contact the
>>> libFuzzer developers.
>>>>> 
>>>>> From the build logs, it looks like we're now building objects with these
>>>>> sanitizer flags:
>>>>> 
>>>>> -fsanitize=address
>>>>> -fsanitize-address-use-after-scope
>>>>> -fsanitize=fuzzer-no-link
>>>>> 
>>>>> We're then linking the fuzzer binaries with these:
>>>>> 
>>>>> -fsanitize=address
>>>>> -fsanitize-address-use-after-scope
>>>>> -fsanitize=fuzzer-no-link
>>>>> -fsanitize=fuzzer
>>>>> 
>>>>> Any idea what's wrong or where to start looking?
>>>