[llvm] r270632 - [libfuzzer] Trying random unit prefixes during corpus load.
Mike Aizatsky via llvm-commits
llvm-commits at lists.llvm.org
Wed May 25 13:42:13 PDT 2016
http://reviews.llvm.org/D20641
On Wed, May 25, 2016 at 11:57 AM Kostya Serebryany <kcc at google.com> wrote:
> Strangely, it does not fail on the bot, but still fails for me locally (I
> don't have local modifications, checked).
> Also, if I simply run ./lib/Fuzzer/test/LLVMFuzzer-Unittest I get:
> lib/Fuzzer/FuzzerLoop.cpp:154: fuzzer::Fuzzer::Fuzzer(UserCallback,
> fuzzer::MutationDispatcher &, fuzzer::Fuzzer::FuzzingOptions): Assertion
> `!F' failed.
>
> Anyway, I think using a unit test here is wrong, we need a end-to-end
> tests here instead.
> We already have some such tests -- grep for mkdir in lib/Fuzzer/test
> A test would create a corpus with a large element, then make sure that a
> certain small element (specific sha1sum) is present after several runs.
> Please do this.
>
>
>
>
> On Tue, May 24, 2016 at 5:25 PM, Kostya Serebryany <kcc at google.com> wrote:
>
>> Tests are failing now, please check.
>>
>> [ RUN ] Corpus.TruncateUnits
>> /usr/local/google/home/kcc/llvm/lib/Fuzzer/test/FuzzerUnittest.cpp:445:
>> Failure
>> Value of: NewCorpus.size()
>> Actual: 3
>> Expected: 1ul
>> Which is: 1
>> [ FAILED ] Corpus.TruncateUnits (10 ms)
>> [----------] 1 test from Corpus (11 ms total)
>>
>>
>> On Tue, May 24, 2016 at 4:14 PM, Mike Aizatsky via llvm-commits <
>> llvm-commits at lists.llvm.org> wrote:
>>
>>> Author: aizatsky
>>> Date: Tue May 24 18:14:29 2016
>>> New Revision: 270632
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=270632&view=rev
>>> Log:
>>> [libfuzzer] Trying random unit prefixes during corpus load.
>>>
>>> Differential Revision: http://reviews.llvm.org/D20301
>>>
>>> Modified:
>>> llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
>>> llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>>> llvm/trunk/lib/Fuzzer/FuzzerInternal.h
>>> llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp
>>> llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp
>>>
>>> Modified: llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp?rev=270632&r1=270631&r2=270632&view=diff
>>>
>>> ==============================================================================
>>> --- llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp (original)
>>> +++ llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp Tue May 24 18:14:29 2016
>>> @@ -330,6 +330,7 @@ static int FuzzerDriver(const std::vecto
>>> Options.SaveArtifacts = !DoPlainRun;
>>> Options.PrintNewCovPcs = Flags.print_new_cov_pcs;
>>> Options.PrintFinalStats = Flags.print_final_stats;
>>> + Options.TruncateUnits = Flags.truncate_units;
>>>
>>> unsigned Seed = Flags.seed;
>>> // Initialize Seed.
>>>
>>> Modified: llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerFlags.def?rev=270632&r1=270631&r2=270632&view=diff
>>>
>>> ==============================================================================
>>> --- llvm/trunk/lib/Fuzzer/FuzzerFlags.def (original)
>>> +++ llvm/trunk/lib/Fuzzer/FuzzerFlags.def Tue May 24 18:14:29 2016
>>> @@ -84,6 +84,7 @@ FUZZER_FLAG_INT(detect_leaks, 1, "If 1,
>>> "try to detect memory leaks during fuzzing (i.e. not only at shut
>>> down).")
>>> FUZZER_FLAG_INT(rss_limit_mb, 2048, "If non-zero, the fuzzer will exit
>>> upon"
>>> "reaching this limit of RSS memory usage.")
>>> +FUZZER_FLAG_INT(truncate_units, 0, "Try truncated units when loading
>>> corpus.")
>>>
>>> FUZZER_DEPRECATED_FLAG(exit_on_first)
>>> FUZZER_DEPRECATED_FLAG(save_minimized_corpus)
>>>
>>> Modified: llvm/trunk/lib/Fuzzer/FuzzerInternal.h
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerInternal.h?rev=270632&r1=270631&r2=270632&view=diff
>>>
>>> ==============================================================================
>>> --- llvm/trunk/lib/Fuzzer/FuzzerInternal.h (original)
>>> +++ llvm/trunk/lib/Fuzzer/FuzzerInternal.h Tue May 24 18:14:29 2016
>>> @@ -317,6 +317,7 @@ public:
>>> bool PrintNewCovPcs = false;
>>> bool PrintFinalStats = false;
>>> bool DetectLeaks = true;
>>> + bool TruncateUnits = false;
>>> };
>>>
>>> // Aggregates all available coverage measurements.
>>> @@ -354,6 +355,7 @@ public:
>>> }
>>> size_t ChooseUnitIdxToMutate();
>>> const Unit &ChooseUnitToMutate() { return
>>> Corpus[ChooseUnitIdxToMutate()]; };
>>> + void TruncateUnits(std::vector<Unit> *NewCorpus);
>>> void Loop();
>>> void Drill();
>>> void ShuffleAndMinimize();
>>> @@ -396,6 +398,9 @@ public:
>>> void SetMaxLen(size_t MaxLen);
>>> void RssLimitCallback();
>>>
>>> + // Public for tests.
>>> + void ResetCoverage();
>>> +
>>> private:
>>> void AlarmCallback();
>>> void CrashCallback();
>>> @@ -416,7 +421,6 @@ private:
>>> // Must be called whenever the corpus or unit weights are changed.
>>> void UpdateCorpusDistribution();
>>>
>>> - void ResetCoverage();
>>> bool UpdateMaxCoverage();
>>>
>>> // Trace-based fuzzing: we run a unit with some kind of tracing
>>>
>>> Modified: llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp?rev=270632&r1=270631&r2=270632&view=diff
>>>
>>> ==============================================================================
>>> --- llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp (original)
>>> +++ llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp Tue May 24 18:14:29 2016
>>> @@ -59,6 +59,7 @@ __attribute__((weak)) int __lsan_do_reco
>>>
>>> namespace fuzzer {
>>> static const size_t kMaxUnitSizeToPrint = 256;
>>> +static const size_t TruncateMaxRuns = 1000;
>>>
>>> static void MissingWeakApiFunction(const char *FnName) {
>>> Printf("ERROR: %s is not defined. Exiting.\n"
>>> @@ -353,12 +354,54 @@ void Fuzzer::ShuffleCorpus(UnitVector *V
>>> });
>>> }
>>>
>>> +// Tries random prefixes of corpus items.
>>> +// Prefix length is chosen according to exponential distribution
>>> +// to sample short lengths much more heavily.
>>> +void Fuzzer::TruncateUnits(std::vector<Unit> *NewCorpus) {
>>> + size_t MaxCorpusLen = 0;
>>> + for (const auto &U : Corpus)
>>> + MaxCorpusLen = std::max(MaxCorpusLen, U.size());
>>> +
>>> + if (MaxCorpusLen <= 1)
>>> + return;
>>> +
>>> + // 50% of exponential distribution is Log[2]/lambda.
>>> + // Choose lambda so that median is MaxCorpusLen / 2.
>>> + double Lambda = 2.0 * log(2.0) / static_cast<double>(MaxCorpusLen);
>>> + std::exponential_distribution<> Dist(Lambda);
>>> + std::vector<double> Sizes;
>>> + size_t TruncatePoints = std::max(1ul, TruncateMaxRuns /
>>> Corpus.size());
>>> + Sizes.reserve(TruncatePoints);
>>> + for (size_t I = 0; I < TruncatePoints; ++I) {
>>> + Sizes.push_back(Dist(MD.GetRand().Get_mt19937()) + 1);
>>> + }
>>> + std::sort(Sizes.begin(), Sizes.end());
>>> +
>>> + for (size_t S : Sizes) {
>>> + for (const auto &U : Corpus) {
>>> + if (S < U.size() && RunOne(U.data(), S)) {
>>> + Unit U1(U.begin(), U.begin() + S);
>>> + NewCorpus->push_back(U1);
>>> + WriteToOutputCorpus(U1);
>>> + PrintStatusForNewUnit(U1);
>>> + }
>>> + }
>>> + }
>>> + PrintStats("TRUNC ");
>>> +}
>>> +
>>> void Fuzzer::ShuffleAndMinimize() {
>>> PrintStats("READ ");
>>> std::vector<Unit> NewCorpus;
>>> if (Options.ShuffleAtStartUp)
>>> ShuffleCorpus(&Corpus);
>>>
>>> + if (Options.TruncateUnits) {
>>> + ResetCoverage();
>>> + TruncateUnits(&NewCorpus);
>>> + ResetCoverage();
>>> + }
>>> +
>>> for (const auto &U : Corpus) {
>>> if (RunOne(U)) {
>>> NewCorpus.push_back(U);
>>>
>>> Modified: llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp?rev=270632&r1=270631&r2=270632&view=diff
>>>
>>> ==============================================================================
>>> --- llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp (original)
>>> +++ llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp Tue May 24 18:14:29
>>> 2016
>>> @@ -13,6 +13,10 @@ extern "C" int LLVMFuzzerTestOneInput(co
>>> abort();
>>> }
>>>
>>> +static int EmptyLLVMFuzzerTestOneInput(const uint8_t *Data, size_t
>>> Size) {
>>> + return 0;
>>> +}
>>> +
>>> TEST(Fuzzer, CrossOver) {
>>> Random Rand(0);
>>> MutationDispatcher MD(Rand);
>>> @@ -423,3 +427,21 @@ TEST(Corpus, Distribution) {
>>> EXPECT_GT(Hist[i], TriesPerUnit / N / 3);
>>> }
>>> }
>>> +
>>> +TEST(Corpus, TruncateUnits) {
>>> + Random Rand(0);
>>> + MutationDispatcher MD(Rand);
>>> + Fuzzer::FuzzingOptions Options;
>>> + Options.OutputCorpus = ""; // stops from writing new units.
>>> + Fuzzer Fuzz(EmptyLLVMFuzzerTestOneInput, MD, Options);
>>> +
>>> + Fuzz.AddToCorpus(Unit(1024, static_cast<uint8_t>(1)));
>>> + Fuzz.ResetCoverage();
>>> +
>>> + std::vector<Unit> NewCorpus;
>>> + Fuzz.TruncateUnits(&NewCorpus);
>>> +
>>> + // New corpus should have a shorter unit.
>>> + EXPECT_EQ(1ul, NewCorpus.size());
>>> + EXPECT_EQ(1ul, NewCorpus[0].size());
>>> +}
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160525/d333bfdf/attachment.html>
More information about the llvm-commits
mailing list