<div dir="ltr"><div><div><div><div><div>Hi, Kostya,<br><br></div>I converted 62 existing tests in linux:samples/bpf/test_verifier.c as the seed test cases<br></div>in corpus. Indeed, I got a good initial jump of coverage (edge,8bit-counters) compared to<br></div>non corpus. The change is at<br><span><span><a href="https://github.com/iovisor/bpf-fuzzer" rel="noreferrer" target="_blank">https://github.com/iovisor/bpf-fuzzer</a><br><br></span></span></div><span><span>I am using a virtual machine on my laptop and in the fast two days it may have run<br></span></span></div><span><span>on-off for some times. Last screen shot before I have to reboot the machine is:<br><br>#3284486954 NEW cov: 883 bits: 3739 units: 928 exec/s: 38510 L: 55<br>#3294370138 NEW cov: 883 bits: 3742 units: 929 exec/s: 38495 L: 62<br>#3309856564 NEW cov: 883 bits: 3743 units: 930 exec/s: 38484 L: 63<br>#3311967702 NEW cov: 883 bits: 3744 units: 931 exec/s: 38483 L: 63<br>#3312177180 NEW cov: 883 bits: 3745 units: 932 exec/s: 38482 L: 64<br>#3424548361 NEW cov: 883 bits: 3746 units: 933 exec/s: 38426 L: 61<br>#3450807353 NEW cov: 883 bits: 3747 units: 934 exec/s: 38412 L: 49<br>#3460220158 NEW cov: 884 bits: 3748 units: 935 exec/s: 38410 L: 47<br>#3466257868 NEW cov: 884 bits: 3749 units: 936 exec/s: 38408 L: 47<br>#3486994800 NEW cov: 884 bits: 3750 units: 937 exec/s: 38396 L: 64<br>#3491234878 NEW cov: 886 bits: 3752 units: 938 exec/s: 38394 L: 58<br>#3495108828 NEW cov: 886 bits: 3754 units: 939 exec/s: 38391 L: 58<br><br></span></span><div><div><div><div><div>The corpus itself has 1000 test cases including 62 initial ones.<br><br></div><div>I do not know the total edge blocks in the code to assess how much room is left<br></div><div>for coverage testing. A rough annotation with source code to find which parts of<br>source codes are covered will be really helpful. Any support in llvm for this?<br><br>Also memory leak may not be tested in this setup since it does not quit.<br></div><div><br></div><div>Thanks,<br><br></div><div>Yonghong<br></div><div><div><div><div class="gmail_extra"><br><div class="gmail_quote">[ I changed my email address from company one to my personal address<br></div><div class="gmail_quote"> for which I have subscription for various llvm aliases.]<br></div><div class="gmail_quote"><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Kostya Serebryany</b> <span dir="ltr"><<a href="mailto:kcc@google.com" target="_blank">kcc@google.com</a>></span><br>Date: Fri, Sep 11, 2015 at 1:12 PM<br>Subject: Re: [llvm] r247425 - [libFuzzer] mention more trophies<br>To: Yonghong Song <<a href="mailto:yhs@plumgrid.com" target="_blank">yhs@plumgrid.com</a>><br>Cc: Alexei Starovoitov <<a href="mailto:alexei.starovoitov@gmail.com" target="_blank">alexei.starovoitov@gmail.com</a>>, LLVM Commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>><br><br><br><div dir="ltr">You may also want to add "8bit-counters" to -fsanitize-coverage=...</div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Sep 11, 2015 at 10:36 AM, Kostya Serebryany <span dir="ltr"><<a href="mailto:kcc@google.com" target="_blank">kcc@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Fri, Sep 11, 2015 at 10:14 AM, Yonghong Song <span dir="ltr"><<a href="mailto:yhs@plumgrid.com" target="_blank">yhs@plumgrid.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Currently, it starts with an empty corpus. One option may be converting the existing test_verifier test cases into corpus so that fuzzer<div>can start with better initial coverage. </div></div></blockquote><div><br></div></span><div>Yes, I would certainly start from that. </div><div>Please let me know how it goes. </div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I have not experimented this yet. Any suggestions?</div></div><div class="gmail_extra"><div><div><br><div class="gmail_quote">On Fri, Sep 11, 2015 at 10:09 AM, Kostya Serebryany <span dir="ltr"><<a href="mailto:kcc@google.com" target="_blank">kcc@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Fri, Sep 11, 2015 at 10:07 AM, Alexei Starovoitov <span dir="ltr"><<a href="mailto:alexei.starovoitov@gmail.com" target="_blank">alexei.starovoitov@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>On Fri, Sep 11, 2015 at 9:34 AM, Kostya Serebryany via llvm-commits<br>
<<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br>
> +<br>
> +* Linux Kernel's BPF verifier: <a href="https://github.com/iovisor/bpf-fuzzer" rel="noreferrer" target="_blank">https://github.com/iovisor/bpf-fuzzer</a><br>
<br>
</span>yep :)<br>
It found one bug so far, but looks like we need custom<br>
instruction generation. Pure random fuzzing cannot generate<br>
long enough instruction sequences to stress all pieces of verifier.<br></blockquote><div><br></div></div></div><div>Did you fuzz starting from an empty corpus, or you gave it something to start with? </div></div><br></div></div>
</blockquote></div><br><br clear="all"><span class=""><font color="#888888"><div><br></div></font></span></div></div><span class=""><font color="#888888"><span><font color="#888888">-- <br><div><div dir="ltr">Yonghong<div><a href="http://www.plumgrid.com/plumgrid-ignition/" target="_blank">http://www.plumgrid.com/plumgrid-ignition/</a></div></div></div>
</font></span></font></span></div><span class=""><font color="#888888">
</font></span></blockquote></span></div><span class=""><font color="#888888"><br></font></span></div></div><span class=""><font color="#888888">
</font></span></blockquote></div><span class=""><font color="#888888"><br></font></span></div><span class=""><font color="#888888">
</font></span></div></div></div><span class=""><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div><div dir="ltr">Yonghong<div><a href="http://www.plumgrid.com/plumgrid-ignition/" target="_blank">http://www.plumgrid.com/plumgrid-ignition/</a></div></div></div>
</font></span></div>
</blockquote></div><br></div></div></div></div></div></div></div></div></div>