[llvm] r247425 - [libFuzzer] mention more trophies

Thu Sep 17 17:48:30 PDT 2015

On Thu, Sep 17, 2015 at 12:01 PM, Y Song <ys114321 at gmail.com> wrote:

> I am able to use the following commands to increase test input length and
> limit the number of runs
> so that I will get a memory leak report.
>
> ./test_fuzzer -max_len=1024 -runs=1000000 ./corpus
>

Exactly!

>
> Further, use the following command
> objdump -D test_fuzzer | grep '<__sanitizer_cov>' | wc
>

Yep!
You may also want to know exactly which edges are not covered.
There is a handy python script that does this for you:
http://clang.llvm.org/docs/SanitizerCoverage.html#how-good-is-the-coverage
 (no good visualization yet, just the raw info)

--kcc

> I am able to get the number of coverage edges, which is 1106.
>
> Considering my last run reaches:
> #3495108828    NEW    cov: 886 bits: 3754 units: 939 exec/s: 38391 L: 58
>
> Not bad.
>
> On Thu, Sep 17, 2015 at 9:05 AM, Y Song <ys114321 at gmail.com> wrote:
>
>> Hi, Kostya,
>>
>> I converted 62 existing tests in linux:samples/bpf/test_verifier.c as the
>> seed test cases
>> in corpus. Indeed, I got a good initial jump of coverage
>> (edge,8bit-counters) compared to
>> non corpus. The change is at
>> https://github.com/iovisor/bpf-fuzzer
>>
>> I am using a virtual machine on my laptop and in the fast two days it may
>> have run
>> on-off for some times. Last screen shot before I have to reboot the
>> machine is:
>>
>> #3284486954    NEW    cov: 883 bits: 3739 units: 928 exec/s: 38510 L: 55
>> #3294370138    NEW    cov: 883 bits: 3742 units: 929 exec/s: 38495 L: 62
>> #3309856564    NEW    cov: 883 bits: 3743 units: 930 exec/s: 38484 L: 63
>> #3311967702    NEW    cov: 883 bits: 3744 units: 931 exec/s: 38483 L: 63
>> #3312177180    NEW    cov: 883 bits: 3745 units: 932 exec/s: 38482 L: 64
>> #3424548361    NEW    cov: 883 bits: 3746 units: 933 exec/s: 38426 L: 61
>> #3450807353    NEW    cov: 883 bits: 3747 units: 934 exec/s: 38412 L: 49
>> #3460220158    NEW    cov: 884 bits: 3748 units: 935 exec/s: 38410 L: 47
>> #3466257868    NEW    cov: 884 bits: 3749 units: 936 exec/s: 38408 L: 47
>> #3486994800    NEW    cov: 884 bits: 3750 units: 937 exec/s: 38396 L: 64
>> #3491234878    NEW    cov: 886 bits: 3752 units: 938 exec/s: 38394 L: 58
>> #3495108828    NEW    cov: 886 bits: 3754 units: 939 exec/s: 38391 L: 58
>>
>> The corpus itself has 1000 test cases including 62 initial ones.
>>
>> I do not know the total edge blocks in the code to assess how much room
>> is left
>> for coverage testing. A rough annotation with source code to find which
>> parts of
>> source codes are covered will be really helpful. Any support in llvm for
>> this?
>>
>> Also memory leak may not be tested in this setup since it does not quit.
>>
>> Thanks,
>>
>> Yonghong
>>
>> [ I changed my email address from company one to my personal address
>>   for which I have subscription for various llvm aliases.]
>>
>>
>>> ---------- Forwarded message ----------
>>> From: Kostya Serebryany <kcc at google.com>
>>> Date: Fri, Sep 11, 2015 at 1:12 PM
>>> Subject: Re: [llvm] r247425 - [libFuzzer] mention more trophies
>>> To: Yonghong Song <yhs at plumgrid.com>
>>> Cc: Alexei Starovoitov <alexei.starovoitov at gmail.com>, LLVM Commits <
>>> llvm-commits at lists.llvm.org>
>>>
>>>
>>> You may also want to add "8bit-counters" to -fsanitize-coverage=...
>>>
>>> On Fri, Sep 11, 2015 at 10:36 AM, Kostya Serebryany <kcc at google.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Sep 11, 2015 at 10:14 AM, Yonghong Song <yhs at plumgrid.com>
>>>> wrote:
>>>>
>>>>> Currently, it starts with an empty corpus. One option may be
>>>>> converting the existing test_verifier test cases into corpus so that fuzzer
>>>>> can start with better initial coverage.
>>>>>
>>>>
>>>> Yes, I would certainly start from that.
>>>> Please let me know how it goes.
>>>>
>>>>
>>>>> I have not experimented this yet. Any suggestions?
>>>>>
>>>>> On Fri, Sep 11, 2015 at 10:09 AM, Kostya Serebryany <kcc at google.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 11, 2015 at 10:07 AM, Alexei Starovoitov <
>>>>>> alexei.starovoitov at gmail.com> wrote:
>>>>>>
>>>>>>> On Fri, Sep 11, 2015 at 9:34 AM, Kostya Serebryany via llvm-commits
>>>>>>> <llvm-commits at lists.llvm.org> wrote:
>>>>>>> > +
>>>>>>> > +* Linux Kernel's BPF verifier:
>>>>>>> https://github.com/iovisor/bpf-fuzzer
>>>>>>>
>>>>>>> yep :)
>>>>>>> It found one bug so far, but looks like we need custom
>>>>>>> instruction generation. Pure random fuzzing cannot generate
>>>>>>> long enough instruction sequences to stress all pieces of verifier.
>>>>>>>
>>>>>>
>>>>>> Did you fuzz starting from an empty corpus, or you gave it something
>>>>>> to start with?
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Yonghong
>>>>> http://www.plumgrid.com/plumgrid-ignition/
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Yonghong
>>> http://www.plumgrid.com/plumgrid-ignition/
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150917/2000e5a2/attachment.html>