[llvm-dev] Fuzzing bitcode reader

[Kostya Serebryany via llvm-dev]

On Wed, Feb 1, 2017 at 9:19 AM, Michael Kruse <llvmdev at meinersbur.de> wrote:

> 2017-02-01 18:07 GMT+01:00 Kostya Serebryany <kcc at google.com>:
> > Yes, I used to run clang-fuzzer and clang-format-fuzzer on this bot, but
> not
> > any more.
> > The reason is simple -- the bot was always red (well, orange) and the
> bugs
> > were never fixed.
> >
> > Currently we run clang-fuzzer (but not clang-format-fuzzer) on our
> internal
> > fuzzing infra
> > and Richard has fixed at least one bug found this way.
> > http://llvm.org/viewvc/llvm-project?view=revision&revision=291030
> >
> > My llvm fuzzing bot was pretty naive and simple.
> > If we want proper continuous fuzzing for parts of LLVM we either need to
> > build a separate "real" continuous fuzzing process,
> > or use an existing one. Luckily, there is one :)
> > As a pilot I've recently added the cxa_demangler_fuzzer to OSS-Fuzz:
> > https://github.com/google/oss-fuzz/tree/master/projects/llvm_libcxxabi
> > It even found one bug which Mehdi already fixed!
> > http://llvm.org/viewvc/llvm-project?view=revision&revision=293330
> > The bug report itself will become public in ~4 days:
> > https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=370
>
> Thanks for the explanation.
>
>
>
> >> > Another (obvious?) fuzzing candidate would be the LLVM's bitcode
> >> > reader. I ran afl-fuzz on it and it found lots of failed assertions
> >> > within seconds. Isn't fuzzing done on a regular basis as [1] suggests
> >> > should be done? Should I report the crashes found by it?
> >>
> >> The bitcode reader is known to not be robust against malformed inputs.
> >
> >
> > Yes, I afraid the bitcode reader (as some other parts of LLVM) are not
> > robust enough to withstand fuzzing. :(
> > Note that if we want to use libFuzzer (which is an in-process fuzzer) the
> > target should not assert/abort/exit on any input (if it's not a bug).
>
> Is there any incentive to change that?


Not that I know of.


> A google Summer of Code project maybe?
>

Maybe.
The bottleneck is not bug finding, but bug fixing, which sometimes may
require large changes.
And doing code review for such changes might be more work than just making
them.

--kcc

>
> Michael
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170201/cc8d21b7/attachment.html>