[llvm-dev] Fuzzing bitcode reader

[Mehdi Amini via llvm-dev]

> On Feb 1, 2017, at 9:27 AM, Kostya Serebryany <kcc at google.com> wrote:
> 
> 
> 
> On Wed, Feb 1, 2017 at 9:19 AM, Michael Kruse <llvmdev at meinersbur.de <mailto:llvmdev at meinersbur.de>> wrote:
> 2017-02-01 18:07 GMT+01:00 Kostya Serebryany <kcc at google.com <mailto:kcc at google.com>>:
> > Yes, I used to run clang-fuzzer and clang-format-fuzzer on this bot, but not
> > any more.
> > The reason is simple -- the bot was always red (well, orange) and the bugs
> > were never fixed.
> >
> > Currently we run clang-fuzzer (but not clang-format-fuzzer) on our internal
> > fuzzing infra
> > and Richard has fixed at least one bug found this way.
> > http://llvm.org/viewvc/llvm-project?view=revision&revision=291030 <http://llvm.org/viewvc/llvm-project?view=revision&revision=291030>
> >
> > My llvm fuzzing bot was pretty naive and simple.
> > If we want proper continuous fuzzing for parts of LLVM we either need to
> > build a separate "real" continuous fuzzing process,
> > or use an existing one. Luckily, there is one :)
> > As a pilot I've recently added the cxa_demangler_fuzzer to OSS-Fuzz:
> > https://github.com/google/oss-fuzz/tree/master/projects/llvm_libcxxabi <https://github.com/google/oss-fuzz/tree/master/projects/llvm_libcxxabi>
> > It even found one bug which Mehdi already fixed!
> > http://llvm.org/viewvc/llvm-project?view=revision&revision=293330 <http://llvm.org/viewvc/llvm-project?view=revision&revision=293330>
> > The bug report itself will become public in ~4 days:
> > https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=370 <https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=370>
> 
> Thanks for the explanation.
> 
> 
> 
> >> > Another (obvious?) fuzzing candidate would be the LLVM's bitcode
> >> > reader. I ran afl-fuzz on it and it found lots of failed assertions
> >> > within seconds. Isn't fuzzing done on a regular basis as [1] suggests
> >> > should be done? Should I report the crashes found by it?
> >>
> >> The bitcode reader is known to not be robust against malformed inputs.
> >
> >
> > Yes, I afraid the bitcode reader (as some other parts of LLVM) are not
> > robust enough to withstand fuzzing. :(
> > Note that if we want to use libFuzzer (which is an in-process fuzzer) the
> > target should not assert/abort/exit on any input (if it's not a bug).
> 
> Is there any incentive to change that? 
> 
> Not that I know of.
>  
> A google Summer of Code project maybe?
> 
> Maybe. 
> The bottleneck is not bug finding, but bug fixing, which sometimes may require large changes. 
> And doing code review for such changes might be more work than just making them. 
> 

For the bitcode for example, I wouldn’t expect it to be large changes that would be complicated to review. However these are still tedious bugs to fix.
About a GSOC, my own personal opinion is that we should try to give interesting / fun projects to student and not use them as cheap labor to fix the small bugs and issues we’re not able to prioritize ourself.

My 2 cents :)

— 
Mehdi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170201/a9ba04f7/attachment.html>