[PATCH] D73776: Entropic: Boosting LibFuzzer Performance

[Max Moroz via Phabricator via llvm-commits]

Dor1s added inline comments.


================
Comment at: compiler-rt/lib/fuzzer/FuzzerCorpus.h:278
+      // Remove most abundant rare feature.
+      RareFeatures.erase(remove(RareFeatures.begin(), RareFeatures.end(),
+                                ST_mostAbundantRareFeatureIdx),
----------------
marcel wrote:
> Dor1s wrote:
> > assuming this code gets executed quite often, and the order inside `RareFeatures` isn't important, we can avoid erase-remove and do something like:
> > 
> > ```
> > RareFeatures[index_from_the_loop] = RareFeatures.back();
> > RareFeatures.resize(RareFeatures.size() - 1);
> > ```
> > 
> > but the loop on line 269 would have to use index in the vector (from 1 to `< RareFeatures.size()`) instead of the iterator
> > 
> > feel free to ignore though, it's just a suggestion which may or may not be a good one :)
> With the subsequent push_back (Line 292), do you mean a swap and pop_back here?
yes, `swap` and `pop_back` would have the same effect


================
Comment at: compiler-rt/lib/fuzzer/FuzzerCorpus.h:380
+  // of the seed. Since we do not know the entropy of a seed that has
+  // never been executed we assign fresh seeds maximum entropy and
+  // let II->Energy approach the true entropy from above.
----------------
marcel wrote:
> Dor1s wrote:
> > From the code below it seems like `Energy` represents entropy and the max value is 0, which we reduce depending on the actual feature frequencies. Is that correct understanding?
> Yes, we estimate the entropy over the probabilities of the features in the neighborhood of the seed. Entropy is positive. The maximum entropy is `logl(GlobalNumberOfFeatures)`.
sorry, I don't understand. Below are the code lines changing `Energy` value:

```
    II->Energy = 0.0;
    II->SumIncidence = 0;

    // Apply add-one smoothing to locally discovered features.
    for (auto F : II->FeatureFreqs) {
      size_t LocalIncidence = F.second + 1;
      Energy -= LocalIncidence * logl(LocalIncidence);
      SumIncidence += LocalIncidence;
    }

    <...>

    // Add a single locally abundant feature apply add-one smoothing.
    size_t AbdIncidence = II->NumExecutedMutations + 1;
    Energy -= AbdIncidence * logl(AbdIncidence);
    <...>

    // Normalize.
    if (SumIncidence != 0)
      Energy = (Energy / SumIncidence) + logl(SumIncidence);

    II->Energy = (double)Energy;
  <...>
  }
```

as I read this, I see that `Energy` should be negative in many cases?



CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D73776/new/

https://reviews.llvm.org/D73776