[llvm-dev] -sanitizer-coverage-prune-blocks=true and LibFuzzer

[Jonas Wagner via llvm-dev]

Hello LLVM devs,

I'm running lots of experiments with LibFuzzer these days -- it's an
amazing tool!

I've noticed something weird while examining the effect of various coverage
options: for one of my benchmarks, the fuzzer was achieving a higher total
coverage before April 2016, when -sanitizer-coverage-prune-blocks became
true by default (commit
<https://github.com/llvm-mirror/llvm/commit/f593646d9ade6ab311cd6c55880bf3bdfc4b26a4>
).

Here are the numbers from running the fuzzer with different options for a
fixed amount of time.

#units  #blocks  #bits   options
492     447      1666    -fsanitize-coverage=edge,indirect-calls,8bit-counters
-mllvm -sanitizer-coverage-prune-blocks=false
135     447      754     -fsanitize-coverage=edge -mllvm
-sanitizer-coverage-prune-blocks=false
 58     103      185
-fsanitize-coverage=edge,indirect-calls,8bit-counters
135     447      754     -fsanitize-coverage=edge

Here, units is the number of test cases generated, blocks is the number of
basic blocks covered (as measured by replaying the corpus using the
-sanitizer-coverage-prune-blocks=false version), and bits is the number of
bits from the 8bit-counters that were seen.

The outlier is the case where I use 8bit-counters and set
-sanitizer-coverage-prune-blocks to true. There are two things I don't
understand: (1) why can the 8bit-counters perform worse than using edge
coverage only, and (2) why does -sanitizer-coverage-prune-blocks affect the
result so much?

Any ideas?

If needed, I can provide scripts to reproduce this (it's a bit of work to
clean them up, though). But maybe someone already has an idea why this
could happen.

Cheers,
Jonas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160920/b91aba2a/attachment.html>