[llvm] r270632 - [libfuzzer] Trying random unit prefixes during corpus load.
Kostya Serebryany via llvm-commits
llvm-commits at lists.llvm.org
Wed May 25 11:57:48 PDT 2016
Strangely, it does not fail on the bot, but still fails for me locally (I
don't have local modifications, checked).
Also, if I simply run ./lib/Fuzzer/test/LLVMFuzzer-Unittest I get:
lib/Fuzzer/FuzzerLoop.cpp:154: fuzzer::Fuzzer::Fuzzer(UserCallback,
fuzzer::MutationDispatcher &, fuzzer::Fuzzer::FuzzingOptions): Assertion
`!F' failed.
Anyway, I think using a unit test here is wrong, we need a end-to-end tests
here instead.
We already have some such tests -- grep for mkdir in lib/Fuzzer/test
A test would create a corpus with a large element, then make sure that a
certain small element (specific sha1sum) is present after several runs.
Please do this.
On Tue, May 24, 2016 at 5:25 PM, Kostya Serebryany <kcc at google.com> wrote:
> Tests are failing now, please check.
>
> [ RUN ] Corpus.TruncateUnits
> /usr/local/google/home/kcc/llvm/lib/Fuzzer/test/FuzzerUnittest.cpp:445:
> Failure
> Value of: NewCorpus.size()
> Actual: 3
> Expected: 1ul
> Which is: 1
> [ FAILED ] Corpus.TruncateUnits (10 ms)
> [----------] 1 test from Corpus (11 ms total)
>
>
> On Tue, May 24, 2016 at 4:14 PM, Mike Aizatsky via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> Author: aizatsky
>> Date: Tue May 24 18:14:29 2016
>> New Revision: 270632
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=270632&view=rev
>> Log:
>> [libfuzzer] Trying random unit prefixes during corpus load.
>>
>> Differential Revision: http://reviews.llvm.org/D20301
>>
>> Modified:
>> llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
>> llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>> llvm/trunk/lib/Fuzzer/FuzzerInternal.h
>> llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp
>> llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp
>>
>> Modified: llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp?rev=270632&r1=270631&r2=270632&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp (original)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp Tue May 24 18:14:29 2016
>> @@ -330,6 +330,7 @@ static int FuzzerDriver(const std::vecto
>> Options.SaveArtifacts = !DoPlainRun;
>> Options.PrintNewCovPcs = Flags.print_new_cov_pcs;
>> Options.PrintFinalStats = Flags.print_final_stats;
>> + Options.TruncateUnits = Flags.truncate_units;
>>
>> unsigned Seed = Flags.seed;
>> // Initialize Seed.
>>
>> Modified: llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerFlags.def?rev=270632&r1=270631&r2=270632&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerFlags.def (original)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerFlags.def Tue May 24 18:14:29 2016
>> @@ -84,6 +84,7 @@ FUZZER_FLAG_INT(detect_leaks, 1, "If 1,
>> "try to detect memory leaks during fuzzing (i.e. not only at shut
>> down).")
>> FUZZER_FLAG_INT(rss_limit_mb, 2048, "If non-zero, the fuzzer will exit
>> upon"
>> "reaching this limit of RSS memory usage.")
>> +FUZZER_FLAG_INT(truncate_units, 0, "Try truncated units when loading
>> corpus.")
>>
>> FUZZER_DEPRECATED_FLAG(exit_on_first)
>> FUZZER_DEPRECATED_FLAG(save_minimized_corpus)
>>
>> Modified: llvm/trunk/lib/Fuzzer/FuzzerInternal.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerInternal.h?rev=270632&r1=270631&r2=270632&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerInternal.h (original)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerInternal.h Tue May 24 18:14:29 2016
>> @@ -317,6 +317,7 @@ public:
>> bool PrintNewCovPcs = false;
>> bool PrintFinalStats = false;
>> bool DetectLeaks = true;
>> + bool TruncateUnits = false;
>> };
>>
>> // Aggregates all available coverage measurements.
>> @@ -354,6 +355,7 @@ public:
>> }
>> size_t ChooseUnitIdxToMutate();
>> const Unit &ChooseUnitToMutate() { return
>> Corpus[ChooseUnitIdxToMutate()]; };
>> + void TruncateUnits(std::vector<Unit> *NewCorpus);
>> void Loop();
>> void Drill();
>> void ShuffleAndMinimize();
>> @@ -396,6 +398,9 @@ public:
>> void SetMaxLen(size_t MaxLen);
>> void RssLimitCallback();
>>
>> + // Public for tests.
>> + void ResetCoverage();
>> +
>> private:
>> void AlarmCallback();
>> void CrashCallback();
>> @@ -416,7 +421,6 @@ private:
>> // Must be called whenever the corpus or unit weights are changed.
>> void UpdateCorpusDistribution();
>>
>> - void ResetCoverage();
>> bool UpdateMaxCoverage();
>>
>> // Trace-based fuzzing: we run a unit with some kind of tracing
>>
>> Modified: llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp?rev=270632&r1=270631&r2=270632&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp (original)
>> +++ llvm/trunk/lib/Fuzzer/FuzzerLoop.cpp Tue May 24 18:14:29 2016
>> @@ -59,6 +59,7 @@ __attribute__((weak)) int __lsan_do_reco
>>
>> namespace fuzzer {
>> static const size_t kMaxUnitSizeToPrint = 256;
>> +static const size_t TruncateMaxRuns = 1000;
>>
>> static void MissingWeakApiFunction(const char *FnName) {
>> Printf("ERROR: %s is not defined. Exiting.\n"
>> @@ -353,12 +354,54 @@ void Fuzzer::ShuffleCorpus(UnitVector *V
>> });
>> }
>>
>> +// Tries random prefixes of corpus items.
>> +// Prefix length is chosen according to exponential distribution
>> +// to sample short lengths much more heavily.
>> +void Fuzzer::TruncateUnits(std::vector<Unit> *NewCorpus) {
>> + size_t MaxCorpusLen = 0;
>> + for (const auto &U : Corpus)
>> + MaxCorpusLen = std::max(MaxCorpusLen, U.size());
>> +
>> + if (MaxCorpusLen <= 1)
>> + return;
>> +
>> + // 50% of exponential distribution is Log[2]/lambda.
>> + // Choose lambda so that median is MaxCorpusLen / 2.
>> + double Lambda = 2.0 * log(2.0) / static_cast<double>(MaxCorpusLen);
>> + std::exponential_distribution<> Dist(Lambda);
>> + std::vector<double> Sizes;
>> + size_t TruncatePoints = std::max(1ul, TruncateMaxRuns / Corpus.size());
>> + Sizes.reserve(TruncatePoints);
>> + for (size_t I = 0; I < TruncatePoints; ++I) {
>> + Sizes.push_back(Dist(MD.GetRand().Get_mt19937()) + 1);
>> + }
>> + std::sort(Sizes.begin(), Sizes.end());
>> +
>> + for (size_t S : Sizes) {
>> + for (const auto &U : Corpus) {
>> + if (S < U.size() && RunOne(U.data(), S)) {
>> + Unit U1(U.begin(), U.begin() + S);
>> + NewCorpus->push_back(U1);
>> + WriteToOutputCorpus(U1);
>> + PrintStatusForNewUnit(U1);
>> + }
>> + }
>> + }
>> + PrintStats("TRUNC ");
>> +}
>> +
>> void Fuzzer::ShuffleAndMinimize() {
>> PrintStats("READ ");
>> std::vector<Unit> NewCorpus;
>> if (Options.ShuffleAtStartUp)
>> ShuffleCorpus(&Corpus);
>>
>> + if (Options.TruncateUnits) {
>> + ResetCoverage();
>> + TruncateUnits(&NewCorpus);
>> + ResetCoverage();
>> + }
>> +
>> for (const auto &U : Corpus) {
>> if (RunOne(U)) {
>> NewCorpus.push_back(U);
>>
>> Modified: llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp?rev=270632&r1=270631&r2=270632&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp (original)
>> +++ llvm/trunk/lib/Fuzzer/test/FuzzerUnittest.cpp Tue May 24 18:14:29 2016
>> @@ -13,6 +13,10 @@ extern "C" int LLVMFuzzerTestOneInput(co
>> abort();
>> }
>>
>> +static int EmptyLLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size)
>> {
>> + return 0;
>> +}
>> +
>> TEST(Fuzzer, CrossOver) {
>> Random Rand(0);
>> MutationDispatcher MD(Rand);
>> @@ -423,3 +427,21 @@ TEST(Corpus, Distribution) {
>> EXPECT_GT(Hist[i], TriesPerUnit / N / 3);
>> }
>> }
>> +
>> +TEST(Corpus, TruncateUnits) {
>> + Random Rand(0);
>> + MutationDispatcher MD(Rand);
>> + Fuzzer::FuzzingOptions Options;
>> + Options.OutputCorpus = ""; // stops from writing new units.
>> + Fuzzer Fuzz(EmptyLLVMFuzzerTestOneInput, MD, Options);
>> +
>> + Fuzz.AddToCorpus(Unit(1024, static_cast<uint8_t>(1)));
>> + Fuzz.ResetCoverage();
>> +
>> + std::vector<Unit> NewCorpus;
>> + Fuzz.TruncateUnits(&NewCorpus);
>> +
>> + // New corpus should have a shorter unit.
>> + EXPECT_EQ(1ul, NewCorpus.size());
>> + EXPECT_EQ(1ul, NewCorpus[0].size());
>> +}
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160525/d5b01dd3/attachment.html>
More information about the llvm-commits
mailing list