[PATCH] D73776: Entropic: Boosting LibFuzzer Performance

[marcel via Phabricator via llvm-commits]

marcel updated this revision to Diff 246591.
marcel marked 3 inline comments as done.
marcel added a comment.

Changes since last commit:

- Better memory footprint
  - Global feature frequencies is a fixed size array with `uint16_t`-width elements: `uint16_t GlobalFeatureFreqs[kFeatureSetSize]` (instead of `size_t`-width)
  - Local feature frequencies is a sorted vector of pairs: `Vector<std::pair<uint32_t,uint16_t>> FeatureFreqs` (instead of unordered `unordered_map<size_t,size_t>`)
- Added options to enable/disable entropic and pulled some constants out:
  - Option `entropic` (default: 0): Enables entropic power schedule.
  - Option `considered_rare` (default: 0xFF): If entropic is enabled, all features which are observed less often than the specified value are considered as rare.
  - Option `topX_rarest_features` (default: 100): If entropic is enabled, we keep track of the frequencies only for the Top-X least abundant features (union features that are considered as rare).
  - Option `sparse_energy_updates` (default: 10000): If entropic is enabled, the inverse value specifies the probability to update the corpus distribution even though no features or seeds were added or deleted. A larger value means less updates.
  - Constant `kProbAgressiveSchedule` (default: 80): Determines the probability to choose a more aggressive schedule (assigning zero weight to below average seeds).
- Turned most occurrences of `long double` to `double`.
- Better variable names and comments.
- Removed some redundant code.
- Fuzzer unit tests (Thanks @Valentin!)

The patch applies to LLVM-trunk. We tested these changes by repeating all our FTS experiments (20 runs).
The performance results look pretty much the same as with the other patch, which is good.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D73776/new/

https://reviews.llvm.org/D73776

Files:
  compiler-rt/lib/fuzzer/FuzzerCorpus.h
  compiler-rt/lib/fuzzer/FuzzerDriver.cpp
  compiler-rt/lib/fuzzer/FuzzerFlags.def
  compiler-rt/lib/fuzzer/FuzzerLoop.cpp
  compiler-rt/lib/fuzzer/FuzzerOptions.h
  compiler-rt/lib/fuzzer/tests/FuzzerUnittest.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D73776.246591.patch
Type: text/x-patch
Size: 19170 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200226/70067db7/attachment.bin>