[llvm-dev] Fuzzing bitcode reader

[Kostya Serebryany via llvm-dev]

On Wed, Feb 1, 2017 at 8:45 AM, Mehdi Amini <mehdi.amini at apple.com> wrote:

>
> > On Feb 1, 2017, at 8:34 AM, Michael Kruse via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > Hi all,
> >
> > The blog entry [1] suggest that one of the buildbots constantly fuzzes
> > clang and clang-format. However, the actual bot [2] only tests the
> > fuzzer itself over a well-known set of bugs in standard software (eg.
> > Heartbleed [3] seems to be among them).
>
> Isn’t it this stage? http://lab.llvm.org:8011/build
> ers/sanitizer-x86_64-linux-fuzzer/builds/2755/steps/
> stage2%2Fasan%2Bassertions%20check-fuzzer/logs/stdio
>
> > Has there actually ever been a
> > buildbot that fuzzes clang/LLVM itself?
>

Yes, I used to run clang-fuzzer and clang-format-fuzzer on this bot, but
not any more.
The reason is simple -- the bot was always red (well, orange) and the bugs
were never fixed.

Currently we run clang-fuzzer (but not clang-format-fuzzer) on our internal
fuzzing infra
and Richard has fixed at least one bug found this way.
http://llvm.org/viewvc/llvm-project?view=revision&revision=291030

My llvm fuzzing bot was pretty naive and simple.
If we want proper continuous fuzzing for parts of LLVM we either need to
build a separate "real" continuous fuzzing process,
or use an existing one. Luckily, there is one :)
As a pilot I've recently added the cxa_demangler_fuzzer to OSS-Fuzz
<https://github.com/google/oss-fuzz>:
https://github.com/google/oss-fuzz/tree/master/projects/llvm_libcxxabi
It even found one bug which Mehdi already fixed!
http://llvm.org/viewvc/llvm-project?view=revision&revision=293330
The bug report itself will become public in ~4 days:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=370

If we want to run some more llvm fuzzers on OSS-Fuzz I'd be happy to (help)
set them up.



> >
> > Another (obvious?) fuzzing candidate would be the LLVM's bitcode
> > reader. I ran afl-fuzz on it and it found lots of failed assertions
> > within seconds. Isn't fuzzing done on a regular basis as [1] suggests
> > should be done? Should I report the crashes found by it?
>
> The bitcode reader is known to not be robust against malformed inputs.
>

Yes, I afraid the bitcode reader (as some other parts of LLVM) are not
robust enough to withstand fuzzing. :(
Note that if we want to use libFuzzer (which is an in-process fuzzer) the
target should not assert/abort/exit on any input (if it's not a bug).

--kcc



>
> —
> Mehdi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170201/33ce1dd8/attachment-0001.html>