[cfe-dev] Introducing clang-triage: A bot to test clang with fuzzed inputs

[John Regehr]

> In the very least I believe it's safe to say the fuzzer is heavily
> biased towards finding parser bugs as long as they are relatively
> common. Whether it can go deeper than that is to be seen.

Yep.  It'll be pretty interesting if that works.

> with llvm 3.5. With CReduce using clang, I don't find it that
> surprising that it fails on some of the inputs that also cause Clang
> to crash.

C-Reduce uses a clang-based tool to do some of its work, but the main 
C-Reduce loop is not based on clang and doesn't care that much if the 
clang part crashes. C-Reduce will work just fine (though of course its 
reduction power will be somewhat degraded) even if the clang tool crashes 
every time.  I advise simply ignoring these crashes, and if you like I can 
add a command line option that causes it to hide the crashes from you.

> A bit hard to see how they would work together. IIUC Csmith tries hard
> to produce code that is also a semantically valid C program;

It does, and afl-fuzz will ruin that property, but that isn't important. 
The important thing is that Csmith will probably cover branches in LLVM 
that the test suite does not cover, and then afl-fuzz can use this as the 
basis for exploring still more code, meaning that Csmith+afl-fuzz may well 
find bugs that are not findable otherwise. I agree that you might as well 
wait for testing based on the test suite to run out of steam before doing 
this, but I wouldn't advise reading too much into whether afl-fuzz empties 
its queue or not, since its queuing policy (as far as I can tell) is more 
or less arbitrary anyway.

John