[llvm-dev] Some feedback on Libfuzzer

[Greg Stark via llvm-dev]

On Sat, Sep 5, 2015 at 6:38 PM, Kostya Serebryany <kcc at google.com> wrote:
>
> This is more like a limitation of asan, not libFuzzer.
> By design, asan does not recover from the first crash.
> This feature has been criticized quite a lot, but I am still convinced this
> is a feature, not a bug.
> IMHO, recovery mode will be misused/abused too often to be useful, besides
> it adds complexity to the code.
> (There is a patch under review right now to implement recovery mode for
> asan,
> but I am not sure if or when this patch will be committed)

Arguably a fuzzer changes the game somewhat. It's one thing to have a
crash during interactive testing but fuzzing is most useful if left to
run on a server unattended.

However I think this isn't the whole story here. There are many
different kinds of errors and not all are unrecoverable memory
failures. I may just get an internal error from my own code that I
know should never happen but doesn't represent major corruption. I
would like a way for the callback to return an error condition to the
fuzzer driver to have the test case set aside as a crash or perhaps as
a separate category (or perhaps to allow it to specify the name of the
category).

>> Also, one more thing, currently Libfuzzer does not catch SIGABRT and
>> treat it as a fatal event. I've added a SIGABRT handler to my own code
>> and moved StaticDeathCallback to public so I can call it from there.
>>
> Again, this is asan, not libFuzzer.
> You need ASAN_OPTIONS=handle_abort=1
> I hope to make it the default soon-ish.

Ah, that would have made this work a bit simpler. Thanks.

I have yet to really experiment with the sanitizers so I don't know if
asan is really doing anything for me given Postgres's internal memory
management. I couldn't get msan running but it sounds more promising
to me. I'll have to try again.

-- 
greg