[compiler-rt] r316382 - [libFuzzer] Periodically purge allocator's quarantine to prolong fuzzing sessions.
Kostya Serebryany via llvm-commits
llvm-commits at lists.llvm.org
Wed Oct 25 16:00:16 PDT 2017
On Wed, Oct 25, 2017 at 3:11 PM, Steven Wu via llvm-commits <
llvm-commits at lists.llvm.org> wrote:
> Hi Alex
>
> This commit seems to break the greendragon libFuzzer bots in a weird way.
> http://green.lab.llvm.org/green/job/libFuzzer/8140/
>
> Personally, I have to rollback to macOS 10.11 to reproduce the issue.
> After your commit, it takes much more runs to trigger the exit
> on fuzzer-customcrossover.test. I am not an expert in fuzzer so I don't
> understand how this commit actually trigger the issue.
>
> Before this commit:
>
> $ fuzzer-customcrossover.test.tmp-CustomCrossOverTest -seed=1 -runs=100000
>
>
Weird.
On linux this test finishes much sooner regardless the seed.
Can you run 1000 times without -seed=1 (i.e. with a random seed) and tell
us the iteration numbers?
> ...
> #80269 REDUCE cov: 38 ft: 64 corp: 17/10063b exec/s: 26756 rss: 306Mb L:
> 961/4035 MS: 2 ChangeBinInt-EraseBytes-
> #97458 NEW cov: 39 ft: 65 corp: 18/10064b exec/s: 32486 rss: 361Mb L:
> 1/4035 MS: 1 ChangeByte-
> BINGO; Found the target, exiting
> ==71874== ERROR: libFuzzer: fuzz target exited
> #0 0x1094e8ee0 in __sanitizer_print_stack_trace asan_stack.cc:38
> #1 0x1081f3c3b in fuzzer::Fuzzer::StaticExitCallback()
> FuzzerLoop.cpp:214
> #2 0x7fff960c7450 in __cxa_finalize_ranges (libsystem_c.dylib:x86_64+
> 0x5f450)
> #3 0x7fff960c7766 in exit (libsystem_c.dylib:x86_64+0x5f766)
> #4 0x1081e3b2b in LLVMFuzzerTestOneInput CustomCrossOverTest.cpp:34
> #5 0x1081f4def in fuzzer::Fuzzer::ExecuteCallback(unsigned char
> const*, unsigned long) FuzzerLoop.cpp:471
> #6 0x1081f479b in fuzzer::Fuzzer::RunOne(unsigned char const*,
> unsigned long, bool, fuzzer::InputInfo*) FuzzerLoop.cpp:399
> #7 0x1081f6641 in fuzzer::Fuzzer::MutateAndTestOne()
> FuzzerLoop.cpp:602
> #8 0x1081f7277 in fuzzer::Fuzzer::Loop(std::__1:
> :vector<std::__1::basic_string<char, std::__1::char_traits<char>,
> std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char,
> std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)
> FuzzerLoop.cpp:710
> #9 0x1081eeece in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned
> char const*, unsigned long)) FuzzerDriver.cpp:738
> #10 0x108214cd2 in main FuzzerMain.cpp:20
> #11 0x7fff8c78d5ac in start (libdyld.dylib:x86_64+0x35ac)
>
> SUMMARY: libFuzzer: fuzz target exited
>
>
> After the commit, I have to use -run=1000000 to trigger:
>
> $ fuzzer-customcrossover.test.tmp-CustomCrossOverTest -seed=1
> -runs=1000000
> ...
> #819960 REDUCE cov: 38 ft: 64 corp: 17/10063b exec/s: 35650 rss: 407Mb L:
> 4035/4035 MS: 3 EraseBytes-CopyPart-CopyPart-
> #837409 NEW cov: 39 ft: 65 corp: 18/10064b exec/s: 34892 rss: 407Mb L:
> 1/4035 MS: 2 ChangeBit-ChangeBit-
> BINGO; Found the target, exiting
> ==72288== ERROR: libFuzzer: fuzz target exited
> #0 0x1025d6ee0 in __sanitizer_print_stack_trace asan_stack.cc:38
> #1 0x1012dea2b in fuzzer::Fuzzer::StaticExitCallback()
> FuzzerLoop.cpp:214
> #2 0x7fff960c7450 in __cxa_finalize_ranges (libsystem_c.dylib:x86_64+
> 0x5f450)
> #3 0x7fff960c7766 in exit (libsystem_c.dylib:x86_64+0x5f766)
> #4 0x1012ce8cb in LLVMFuzzerTestOneInput CustomCrossOverTest.cpp:34
> #5 0x1012dfbdf in fuzzer::Fuzzer::ExecuteCallback(unsigned char
> const*, unsigned long) FuzzerLoop.cpp:471
> #6 0x1012df58b in fuzzer::Fuzzer::RunOne(unsigned char const*,
> unsigned long, bool, fuzzer::InputInfo*) FuzzerLoop.cpp:399
> #7 0x1012e1431 in fuzzer::Fuzzer::MutateAndTestOne()
> FuzzerLoop.cpp:602
> #8 0x1012e2225 in fuzzer::Fuzzer::Loop(std::__1:
> :vector<std::__1::basic_string<char, std::__1::char_traits<char>,
> std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char,
> std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)
> FuzzerLoop.cpp:725
> #9 0x1012d9c8e in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned
> char const*, unsigned long)) FuzzerDriver.cpp:739
> #10 0x1012ffbd2 in main FuzzerMain.cpp:20
> #11 0x7fff8c78d5ac in start (libdyld.dylib:x86_64+0x35ac)
>
> SUMMARY: libFuzzer: fuzz target exited
>
>
> The easiest fix might be to update the test to run a bit longer but I am
> not sure if that is the right fix without understanding what is the
> underlying reason:
> diff --git a/test/fuzzer/fuzzer-customcrossover.test b/test/fuzzer/fuzzer-
> customcrossover.test
> index c32079f45..4b87c2cd7 100644
> --- a/test/fuzzer/fuzzer-customcrossover.test
> +++ b/test/fuzzer/fuzzer-customcrossover.test
> @@ -1,6 +1,6 @@
> RUN: %cpp_compiler %S/CustomCrossOverTest.cpp -o %t-CustomCrossOverTest
>
> -RUN: not %t-CustomCrossOverTest -seed=1 -runs=100000 2>&1
> | FileCheck %s --check-prefix=CHECK_CO
> +RUN: not %t-CustomCrossOverTest -seed=1 -runs=1000000 2>&1
> | FileCheck %s --check-prefix=CHECK_CO
> Disable cross_over, verify that we can't find the target w/o it.
> RUN: %t-CustomCrossOverTest -seed=1 -runs=1000000 -cross_over=0 2>&1
> | FileCheck %s --check-prefix=CHECK_NO_CO
>
> Thanks
>
> Steven
>
>
> On Oct 23, 2017, at 3:04 PM, Alex Shlyapnikov via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
> Author: alekseyshl
> Date: Mon Oct 23 15:04:30 2017
> New Revision: 316382
>
> URL: http://llvm.org/viewvc/llvm-project?rev=316382&view=rev
> Log:
> [libFuzzer] Periodically purge allocator's quarantine to prolong fuzzing
> sessions.
>
> Summary:
> Fuzzing targets that allocate/deallocate a lot of memory tend to consume
> a lot of RSS when ASan quarantine is enabled. Purging quarantine between
> iterations and returning memory to OS keeps RSS down and should not
> reduce the quarantine effectiveness provided the fuzz target does not
> preserve state between iterations (in this case this feature can be turned
> off).
>
> Based on D39153.
>
> Reviewers: vitalybuka
>
> Subscribers: llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D39155
>
> Modified:
> compiler-rt/trunk/lib/fuzzer/FuzzerDriver.cpp
> compiler-rt/trunk/lib/fuzzer/FuzzerExtFunctions.def
> compiler-rt/trunk/lib/fuzzer/FuzzerFlags.def
> compiler-rt/trunk/lib/fuzzer/FuzzerInternal.h
> compiler-rt/trunk/lib/fuzzer/FuzzerLoop.cpp
> compiler-rt/trunk/lib/fuzzer/FuzzerOptions.h
>
> Modified: compiler-rt/trunk/lib/fuzzer/FuzzerDriver.cpp
> URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/
> fuzzer/FuzzerDriver.cpp?rev=316382&r1=316381&r2=316382&view=diff
> ============================================================
> ==================
> --- compiler-rt/trunk/lib/fuzzer/FuzzerDriver.cpp (original)
> +++ compiler-rt/trunk/lib/fuzzer/FuzzerDriver.cpp Mon Oct 23 15:04:30 2017
> @@ -578,6 +578,7 @@ int FuzzerDriver(int *argc, char ***argv
> Options.ReloadIntervalSec = Flags.reload;
> Options.OnlyASCII = Flags.only_ascii;
> Options.DetectLeaks = Flags.detect_leaks;
> + Options.PurgeAllocatorIntervalSec = Flags.purge_allocator_interval;
> Options.TraceMalloc = Flags.trace_malloc;
> Options.RssLimitMb = Flags.rss_limit_mb;
> if (Flags.runs >= 0)
>
> Modified: compiler-rt/trunk/lib/fuzzer/FuzzerExtFunctions.def
> URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/
> fuzzer/FuzzerExtFunctions.def?rev=316382&r1=316381&r2=316382&view=diff
> ============================================================
> ==================
> --- compiler-rt/trunk/lib/fuzzer/FuzzerExtFunctions.def (original)
> +++ compiler-rt/trunk/lib/fuzzer/FuzzerExtFunctions.def Mon Oct 23
> 15:04:30 2017
> @@ -33,6 +33,7 @@ EXT_FUNC(__sanitizer_install_malloc_and_
> (void (*malloc_hook)(const volatile void *, size_t),
> void (*free_hook)(const volatile void *)),
> false);
> +EXT_FUNC(__sanitizer_purge_allocator, void, (), false);
> EXT_FUNC(__sanitizer_print_memory_profile, int, (size_t, size_t), false);
> EXT_FUNC(__sanitizer_print_stack_trace, void, (), true);
> EXT_FUNC(__sanitizer_symbolize_pc, void,
>
> Modified: compiler-rt/trunk/lib/fuzzer/FuzzerFlags.def
> URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/
> fuzzer/FuzzerFlags.def?rev=316382&r1=316381&r2=316382&view=diff
> ============================================================
> ==================
> --- compiler-rt/trunk/lib/fuzzer/FuzzerFlags.def (original)
> +++ compiler-rt/trunk/lib/fuzzer/FuzzerFlags.def Mon Oct 23 15:04:30 2017
> @@ -114,6 +114,10 @@ FUZZER_FLAG_INT(close_fd_mask, 0, "If 1,
> "Be careful, this will also close e.g. asan's stderr/stdout.")
> FUZZER_FLAG_INT(detect_leaks, 1, "If 1, and if LeakSanitizer is enabled "
> "try to detect memory leaks during fuzzing (i.e. not only at shut
> down).")
> +FUZZER_FLAG_INT(purge_allocator_interval, 1, "Purge allocator caches and
> "
> + "quarantines every <N> seconds. When rss_limit_mb is specified (>0), "
> + "purging starts when RSS exceeds 50% of rss_limit_mb. Pass "
> + "purge_allocator_interval=-1 to disable this functionality.")
> FUZZER_FLAG_INT(trace_malloc, 0, "If >= 1 will print all mallocs/frees. "
> "If >= 2 will also print stack traces.")
> FUZZER_FLAG_INT(rss_limit_mb, 2048, "If non-zero, the fuzzer will exit
> upon"
>
> Modified: compiler-rt/trunk/lib/fuzzer/FuzzerInternal.h
> URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/
> fuzzer/FuzzerInternal.h?rev=316382&r1=316381&r2=316382&view=diff
> ============================================================
> ==================
> --- compiler-rt/trunk/lib/fuzzer/FuzzerInternal.h (original)
> +++ compiler-rt/trunk/lib/fuzzer/FuzzerInternal.h Mon Oct 23 15:04:30 2017
> @@ -96,6 +96,7 @@ private:
> void CrashOnOverwrittenData();
> void InterruptCallback();
> void MutateAndTestOne();
> + void PurgeAllocator();
> void ReportNewCoverage(InputInfo *II, const Unit &U);
> void PrintPulseAndReportSlowInput(const uint8_t *Data, size_t Size);
> void WriteToOutputCorpus(const Unit &U);
> @@ -124,6 +125,8 @@ private:
> bool HasMoreMallocsThanFrees = false;
> size_t NumberOfLeakDetectionAttempts = 0;
>
> + system_clock::time_point LastAllocatorPurgeAttemptTime =
> system_clock::now();
> +
> UserCallback CB;
> InputCorpus &Corpus;
> MutationDispatcher &MD;
>
> Modified: compiler-rt/trunk/lib/fuzzer/FuzzerLoop.cpp
> URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/
> fuzzer/FuzzerLoop.cpp?rev=316382&r1=316381&r2=316382&view=diff
> ============================================================
> ==================
> --- compiler-rt/trunk/lib/fuzzer/FuzzerLoop.cpp (original)
> +++ compiler-rt/trunk/lib/fuzzer/FuzzerLoop.cpp Mon Oct 23 15:04:30 2017
> @@ -587,7 +587,7 @@ void Fuzzer::MutateAndTestOne() {
> size_t NewSize = 0;
> NewSize = MD.Mutate(CurrentUnitData, Size, CurrentMaxMutationLen);
> assert(NewSize > 0 && "Mutator returned empty unit");
> - assert(NewSize <= CurrentMaxMutationLen && "Mutator return overisized
> unit");
> + assert(NewSize <= CurrentMaxMutationLen && "Mutator return oversized
> unit");
> Size = NewSize;
> II.NumExecutedMutations++;
> if (RunOne(CurrentUnitData, Size, /*MayDeleteFile=*/true, &II))
> @@ -598,6 +598,25 @@ void Fuzzer::MutateAndTestOne() {
> }
> }
>
> +void Fuzzer::PurgeAllocator() {
> + if (Options.PurgeAllocatorIntervalSec < 0 ||
> + !EF->__sanitizer_purge_allocator) {
> + return;
> + }
> + if (duration_cast<seconds>(system_clock::now() -
> + LastAllocatorPurgeAttemptTime).count() <
> + Options.PurgeAllocatorIntervalSec) {
> + return;
> + }
> +
> + if (Options.RssLimitMb <= 0 ||
> + GetPeakRSSMb() > static_cast<size_t>(Options.RssLimitMb) / 2) {
> + EF->__sanitizer_purge_allocator();
> + }
> +
> + LastAllocatorPurgeAttemptTime = system_clock::now();
> +}
> +
> void Fuzzer::ReadAndExecuteSeedCorpora(const Vector<std::string>
> &CorpusDirs) {
> const size_t kMaxSaneLen = 1 << 20;
> const size_t kMinDefaultLen = 4096;
> @@ -699,6 +718,8 @@ void Fuzzer::Loop(const Vector<std::stri
>
> // Perform several mutations and runs.
> MutateAndTestOne();
> +
> + PurgeAllocator();
> }
>
> PrintStats("DONE ", "\n");
>
> Modified: compiler-rt/trunk/lib/fuzzer/FuzzerOptions.h
> URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/
> fuzzer/FuzzerOptions.h?rev=316382&r1=316381&r2=316382&view=diff
> ============================================================
> ==================
> --- compiler-rt/trunk/lib/fuzzer/FuzzerOptions.h (original)
> +++ compiler-rt/trunk/lib/fuzzer/FuzzerOptions.h Mon Oct 23 15:04:30 2017
> @@ -54,6 +54,7 @@ struct FuzzingOptions {
> bool DumpCoverage = false;
> bool UseClangCoverage = false;
> bool DetectLeaks = true;
> + int PurgeAllocatorIntervalSec = 1;
> int UseFeatureFrequency = false;
> int TraceMalloc = 0;
> bool HandleAbrt = false;
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20171025/64851932/attachment.html>
More information about the llvm-commits
mailing list