<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Oct 25, 2017 at 3:11 PM, Steven Wu via llvm-commits <span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word">Hi Alex<div><br></div><div>This commit seems to break the greendragon libFuzzer bots in a weird way. </div><div><a href="http://green.lab.llvm.org/green/job/libFuzzer/8140/" target="_blank">http://green.lab.llvm.org/<wbr>green/job/libFuzzer/8140/</a></div><div><br></div><div>Personally, I have to rollback to macOS 10.11 to reproduce the issue. After your commit, it takes much more runs to trigger the exit on fuzzer-customcrossover.<wbr>test. I am not an expert in fuzzer so I don't understand how this commit actually trigger the issue.</div><div><br></div><div>Before this commit:</div><div></div><blockquote type="cite"><div>$ fuzzer-customcrossover.test.<wbr>tmp-CustomCrossOverTest -seed=1 -runs=100000</div></blockquote></div></blockquote><div><br></div><div>Weird. </div><div>On linux this test finishes much sooner regardless the seed. </div><div>Can you run 1000 times without -seed=1 (i.e. with a random seed) and tell us the iteration numbers? </div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word"><blockquote type="cite"><div>...</div><div><div>#80269<span class="gmail-m_5950299218738954550Apple-tab-span" style="white-space:pre-wrap"> </span>REDUCE cov: 38 ft: 64 corp: 17/10063b exec/s: 26756 rss: 306Mb L: 961/4035 MS: 2 ChangeBinInt-EraseBytes-</div><div>#97458<span class="gmail-m_5950299218738954550Apple-tab-span" style="white-space:pre-wrap"> </span>NEW cov: 39 ft: 65 corp: 18/10064b exec/s: 32486 rss: 361Mb L: 1/4035 MS: 1 ChangeByte-</div><div>BINGO; Found the target, exiting</div><div>==71874== ERROR: libFuzzer: fuzz target exited</div><div> #0 0x1094e8ee0 in __sanitizer_print_stack_trace <a href="http://asan_stack.cc:38" target="_blank">asan_stack.cc:38</a></div><div> #1 0x1081f3c3b in fuzzer::Fuzzer::<wbr>StaticExitCallback() FuzzerLoop.cpp:214</div><div> #2 0x7fff960c7450 in __cxa_finalize_ranges (libsystem_c.dylib:x86_64+<wbr>0x5f450)</div><div> #3 0x7fff960c7766 in exit (libsystem_c.dylib:x86_64+<wbr>0x5f766)</div><div> #4 0x1081e3b2b in LLVMFuzzerTestOneInput CustomCrossOverTest.cpp:34</div><div> #5 0x1081f4def in fuzzer::Fuzzer::<wbr>ExecuteCallback(unsigned char const*, unsigned long) FuzzerLoop.cpp:471</div><div> #6 0x1081f479b in fuzzer::Fuzzer::RunOne(<wbr>unsigned char const*, unsigned long, bool, fuzzer::InputInfo*) FuzzerLoop.cpp:399</div><div> #7 0x1081f6641 in fuzzer::Fuzzer::<wbr>MutateAndTestOne() FuzzerLoop.cpp:602</div><div> #8 0x1081f7277 in fuzzer::Fuzzer::Loop(std::__1:<wbr>:vector<std::__1::basic_<wbr>string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::<wbr>__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) FuzzerLoop.cpp:710</div><div> #9 0x1081eeece in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) FuzzerDriver.cpp:738</div><div> #10 0x108214cd2 in main FuzzerMain.cpp:20</div><div> #11 0x7fff8c78d5ac in start (libdyld.dylib:x86_64+0x35ac)</div><div><br></div><div>SUMMARY: libFuzzer: fuzz target exited</div></div></blockquote><div><div><br></div><div>After the commit, I have to use -run=1000000 to trigger:</div><div></div><blockquote type="cite"><div>$ fuzzer-customcrossover.test.<wbr>tmp-CustomCrossOverTest -seed=1 -runs=1000000</div><div>...</div><div>#819960<span class="gmail-m_5950299218738954550Apple-tab-span" style="white-space:pre-wrap"> </span>REDUCE cov: 38 ft: 64 corp: 17/10063b exec/s: 35650 rss: 407Mb L: 4035/4035 MS: 3 EraseBytes-CopyPart-CopyPart-<br>#837409<span class="gmail-m_5950299218738954550Apple-tab-span" style="white-space:pre-wrap"> </span>NEW cov: 39 ft: 65 corp: 18/10064b exec/s: 34892 rss: 407Mb L: 1/4035 MS: 2 ChangeBit-ChangeBit-<br>BINGO; Found the target, exiting<br>==72288== ERROR: libFuzzer: fuzz target exited<br> #0 0x1025d6ee0 in __sanitizer_print_stack_trace <a href="http://asan_stack.cc:38" target="_blank">asan_stack.cc:38</a><br> #1 0x1012dea2b in fuzzer::Fuzzer::<wbr>StaticExitCallback() FuzzerLoop.cpp:214<br> #2 0x7fff960c7450 in __cxa_finalize_ranges (libsystem_c.dylib:x86_64+<wbr>0x5f450)<br> #3 0x7fff960c7766 in exit (libsystem_c.dylib:x86_64+<wbr>0x5f766)<br> #4 0x1012ce8cb in LLVMFuzzerTestOneInput CustomCrossOverTest.cpp:34<br> #5 0x1012dfbdf in fuzzer::Fuzzer::<wbr>ExecuteCallback(unsigned char const*, unsigned long) FuzzerLoop.cpp:471<br> #6 0x1012df58b in fuzzer::Fuzzer::RunOne(<wbr>unsigned char const*, unsigned long, bool, fuzzer::InputInfo*) FuzzerLoop.cpp:399<br> #7 0x1012e1431 in fuzzer::Fuzzer::<wbr>MutateAndTestOne() FuzzerLoop.cpp:602<br> #8 0x1012e2225 in fuzzer::Fuzzer::Loop(std::__1:<wbr>:vector<std::__1::basic_<wbr>string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<<wbr>std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) FuzzerLoop.cpp:725<br> #9 0x1012d9c8e in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) FuzzerDriver.cpp:739<br> #10 0x1012ffbd2 in main FuzzerMain.cpp:20<br> #11 0x7fff8c78d5ac in start (libdyld.dylib:x86_64+0x35ac)<br><br>SUMMARY: libFuzzer: fuzz target exited</div></blockquote><div><br></div><div>The easiest fix might be to update the test to run a bit longer but I am not sure if that is the right fix without understanding what is the underlying reason:</div><div><div>diff --git a/test/fuzzer/fuzzer-<wbr>customcrossover.test b/test/fuzzer/fuzzer-<wbr>customcrossover.test</div><div>index c32079f45..4b87c2cd7 100644</div><div>--- a/test/fuzzer/fuzzer-<wbr>customcrossover.test</div><div>+++ b/test/fuzzer/fuzzer-<wbr>customcrossover.test</div><div>@@ -1,6 +1,6 @@</div><div> RUN: %cpp_compiler %S/CustomCrossOverTest.cpp -o %t-CustomCrossOverTest</div><div><br></div><div>-RUN: not %t-CustomCrossOverTest -seed=1 -runs=100000 2>&1 | FileCheck %s --check-prefix=CHECK_CO</div><div>+RUN: not %t-CustomCrossOverTest -seed=1 -runs=1000000 2>&1 | FileCheck %s --check-prefix=CHECK_CO</div><div> Disable cross_over, verify that we can't find the target w/o it.</div><div> RUN: %t-CustomCrossOverTest -seed=1 -runs=1000000 -cross_over=0 2>&1 | FileCheck %s --check-prefix=CHECK_NO_CO</div></div><div><br></div><div>Thanks</div><div><br></div><div>Steven</div><div><div class="gmail-h5"><div><br></div><div><br><blockquote type="cite"><div>On Oct 23, 2017, at 3:04 PM, Alex Shlyapnikov via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:</div><br class="gmail-m_5950299218738954550Apple-interchange-newline"><div><div>Author: alekseyshl<br>Date: Mon Oct 23 15:04:30 2017<br>New Revision: 316382<br><br>URL: <a href="http://llvm.org/viewvc/llvm-project?rev=316382&view=rev" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project?rev=316382&view=rev</a><br>Log:<br>[libFuzzer] Periodically purge allocator's quarantine to prolong fuzzing sessions.<br><br>Summary:<br>Fuzzing targets that allocate/deallocate a lot of memory tend to consume<br>a lot of RSS when ASan quarantine is enabled. Purging quarantine between<br>iterations and returning memory to OS keeps RSS down and should not<br>reduce the quarantine effectiveness provided the fuzz target does not<br>preserve state between iterations (in this case this feature can be turned off).<br><br>Based on D39153.<br><br>Reviewers: vitalybuka<br><br>Subscribers: llvm-commits<br><br>Differential Revision: <a href="https://reviews.llvm.org/D39155" target="_blank">https://reviews.llvm.org/<wbr>D39155</a><br><br>Modified:<br> compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerDriver.cpp<br> compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerExtFunctions.def<br> compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerFlags.def<br> compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerInternal.h<br> compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerLoop.cpp<br> compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerOptions.h<br><br>Modified: compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerDriver.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerDriver.cpp?rev=316382&r1=316381&r2=316382&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerDriver.cpp?rev=<wbr>316382&r1=316381&r2=316382&<wbr>view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerDriver.cpp (original)<br>+++ compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerDriver.cpp Mon Oct 23 15:04:30 2017<br>@@ -578,6 +578,7 @@ int FuzzerDriver(int *argc, char ***argv<br> Options.ReloadIntervalSec = Flags.reload;<br> Options.OnlyASCII = Flags.only_ascii;<br> Options.DetectLeaks = Flags.detect_leaks;<br>+ Options.<wbr>PurgeAllocatorIntervalSec = Flags.purge_allocator_<wbr>interval;<br> Options.TraceMalloc = Flags.trace_malloc;<br> Options.RssLimitMb = Flags.rss_limit_mb;<br> if (Flags.runs >= 0)<br><br>Modified: compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerExtFunctions.def<br>URL: <a href="http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerExtFunctions.def?rev=316382&r1=316381&r2=316382&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerExtFunctions.def?<wbr>rev=316382&r1=316381&r2=<wbr>316382&view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerExtFunctions.def (original)<br>+++ compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerExtFunctions.def Mon Oct 23 15:04:30 2017<br>@@ -33,6 +33,7 @@ EXT_FUNC(__sanitizer_install_<wbr>malloc_and_<br> (void (*malloc_hook)(const volatile void *, size_t),<br> void (*free_hook)(const volatile void *)),<br> false);<br>+EXT_FUNC(__sanitizer_purge_<wbr>allocator, void, (), false);<br> EXT_FUNC(__sanitizer_print_<wbr>memory_profile, int, (size_t, size_t), false);<br> EXT_FUNC(__sanitizer_print_<wbr>stack_trace, void, (), true);<br> EXT_FUNC(__sanitizer_<wbr>symbolize_pc, void,<br><br>Modified: compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerFlags.def<br>URL: <a href="http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerFlags.def?rev=316382&r1=316381&r2=316382&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerFlags.def?rev=<wbr>316382&r1=316381&r2=316382&<wbr>view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerFlags.def (original)<br>+++ compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerFlags.def Mon Oct 23 15:04:30 2017<br>@@ -114,6 +114,10 @@ FUZZER_FLAG_INT(close_fd_mask, 0, "If 1,<br> "Be careful, this will also close e.g. asan's stderr/stdout.")<br> FUZZER_FLAG_INT(detect_leaks, 1, "If 1, and if LeakSanitizer is enabled "<br> "try to detect memory leaks during fuzzing (i.e. not only at shut down).")<br>+FUZZER_FLAG_INT(purge_<wbr>allocator_interval, 1, "Purge allocator caches and "<br>+ "quarantines every <N> seconds. When rss_limit_mb is specified (>0), "<br>+ "purging starts when RSS exceeds 50% of rss_limit_mb. Pass "<br>+ "purge_allocator_interval=-<wbr>1 to disable this functionality.")<br> FUZZER_FLAG_INT(trace_malloc, 0, "If >= 1 will print all mallocs/frees. "<br> "If >= 2 will also print stack traces.")<br> FUZZER_FLAG_INT(rss_limit_mb, 2048, "If non-zero, the fuzzer will exit upon"<br><br>Modified: compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerInternal.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerInternal.h?rev=316382&r1=316381&r2=316382&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerInternal.h?rev=<wbr>316382&r1=316381&r2=316382&<wbr>view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerInternal.h (original)<br>+++ compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerInternal.h Mon Oct 23 15:04:30 2017<br>@@ -96,6 +96,7 @@ private:<br> void CrashOnOverwrittenData();<br> void InterruptCallback();<br> void MutateAndTestOne();<br>+ void PurgeAllocator();<br> void ReportNewCoverage(InputInfo *II, const Unit &U);<br> void PrintPulseAndReportSlowInput(<wbr>const uint8_t *Data, size_t Size);<br> void WriteToOutputCorpus(const Unit &U);<br>@@ -124,6 +125,8 @@ private:<br> bool HasMoreMallocsThanFrees = false;<br> size_t NumberOfLeakDetectionAttempts = 0;<br><br>+ system_clock::time_point LastAllocatorPurgeAttemptTime = system_clock::now();<br>+<br> UserCallback CB;<br> InputCorpus &Corpus;<br> MutationDispatcher &MD;<br><br>Modified: compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerLoop.cpp<br>URL: <a href="http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerLoop.cpp?rev=316382&r1=316381&r2=316382&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerLoop.cpp?rev=<wbr>316382&r1=316381&r2=316382&<wbr>view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerLoop.cpp (original)<br>+++ compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerLoop.cpp Mon Oct 23 15:04:30 2017<br>@@ -587,7 +587,7 @@ void Fuzzer::MutateAndTestOne() {<br> size_t NewSize = 0;<br> NewSize = MD.Mutate(CurrentUnitData, Size, CurrentMaxMutationLen);<br> assert(NewSize > 0 && "Mutator returned empty unit");<br>- assert(NewSize <= CurrentMaxMutationLen && "Mutator return overisized unit");<br>+ assert(NewSize <= CurrentMaxMutationLen && "Mutator return oversized unit");<br> Size = NewSize;<br> II.NumExecutedMutations++;<br> if (RunOne(CurrentUnitData, Size, /*MayDeleteFile=*/true, &II))<br>@@ -598,6 +598,25 @@ void Fuzzer::MutateAndTestOne() {<br> }<br> }<br><br>+void Fuzzer::PurgeAllocator() {<br>+ if (Options.<wbr>PurgeAllocatorIntervalSec < 0 ||<br>+ !EF->__sanitizer_purge_<wbr>allocator) {<br>+ return;<br>+ }<br>+ if (duration_cast<seconds>(<wbr>system_clock::now() -<br>+ <wbr>LastAllocatorPurgeAttemptTime)<wbr>.count() <<br>+ Options.<wbr>PurgeAllocatorIntervalSec) {<br>+ return;<br>+ }<br>+<br>+ if (Options.RssLimitMb <= 0 ||<br>+ GetPeakRSSMb() > static_cast<size_t>(Options.<wbr>RssLimitMb) / 2) {<br>+ EF->__sanitizer_purge_<wbr>allocator();<br>+ }<br>+<br>+ LastAllocatorPurgeAttemptTime = system_clock::now();<br>+}<br>+<br> void Fuzzer::<wbr>ReadAndExecuteSeedCorpora(<wbr>const Vector<std::string> &CorpusDirs) {<br> const size_t kMaxSaneLen = 1 << 20;<br> const size_t kMinDefaultLen = 4096;<br>@@ -699,6 +718,8 @@ void Fuzzer::Loop(const Vector<std::stri<br><br> // Perform several mutations and runs.<br> MutateAndTestOne();<br>+<br>+ PurgeAllocator();<br> }<br><br> PrintStats("DONE ", "\n");<br><br>Modified: compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerOptions.h<br>URL: <a href="http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerOptions.h?rev=316382&r1=316381&r2=316382&view=diff" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/compiler-rt/trunk/lib/<wbr>fuzzer/FuzzerOptions.h?rev=<wbr>316382&r1=316381&r2=316382&<wbr>view=diff</a><br>==============================<wbr>==============================<wbr>==================<br>--- compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerOptions.h (original)<br>+++ compiler-rt/trunk/lib/fuzzer/<wbr>FuzzerOptions.h Mon Oct 23 15:04:30 2017<br>@@ -54,6 +54,7 @@ struct FuzzingOptions {<br> bool DumpCoverage = false;<br> bool UseClangCoverage = false;<br> bool DetectLeaks = true;<br>+ int PurgeAllocatorIntervalSec = 1;<br> int UseFeatureFrequency = false;<br> int TraceMalloc = 0;<br> bool HandleAbrt = false;<br><br><br>______________________________<wbr>_________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-commits</a><br></div></div></blockquote></div><br></div></div></div></div><br>______________________________<wbr>_________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-commits</a><br>
<br></blockquote></div><br></div></div>