<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Aug 11, 2017, at 7:16 PM, Kostya Serebryany <<a href="mailto:kcc@google.com" class="">kcc@google.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><br class="Apple-interchange-newline"><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_quote" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">On Fri, Aug 11, 2017 at 7:05 PM, Vedant Kumar<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:vsk@apple.com" target="_blank" class="">vsk@apple.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">Hi,<br class=""><br class="">You can simplify this a bit if libFuzzer has access to the profiling runtime.<br class=""></blockquote><div class=""><br class=""></div><br class="">libFuzzer should be able to link w/ and w/o -fprofile-instr-generate, so these functions will need to be declared "weak"<br class="">But yes, thanks for the hint.<br class="">Next time I touch this code I'll simplify it as you suggest. <br class="">At this moment it is very experimental and the preliminary results are not promising,<br class=""><div class="">so this code may not survive in the long term. </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div class="gmail-h5"><br class="">> On Aug 11, 2017, at 4:03 PM, Kostya Serebryany via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a>> wrote:<br class="">><br class="">> Author: kcc<br class="">> Date: Fri Aug 11 16:03:22 2017<br class="">> New Revision: 310771<br class="">><br class="">> URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project?rev=310771&view=rev" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-<wbr class="">project?rev=310771&view=rev</a><br class="">> Log:<br class="">> [libFuzzer] experimental support for Clang's coverage (fprofile-instr-generate), Linux-only<br class="">><br class="">> Added:<br class="">> llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerClangCounters.cpp<br class="">> llvm/trunk/lib/Fuzzer/test/<wbr class="">fprofile-instr-generate.test<br class="">> Modified:<br class="">> llvm/trunk/lib/Fuzzer/<wbr class="">CMakeLists.txt<br class="">> llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerDefs.h<br class="">> llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.cpp<br class="">> llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.h<br class="">><br class="">> Modified: llvm/trunk/lib/Fuzzer/<wbr class="">CMakeLists.txt<br class="">> URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/CMakeLists.txt?rev=310771&r1=310770&r2=310771&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-<wbr class="">project/llvm/trunk/lib/Fuzzer/<wbr class="">CMakeLists.txt?rev=310771&r1=<wbr class="">310770&r2=310771&view=diff</a><br class="">> ==============================<wbr class="">==============================<wbr class="">==================<br class="">> --- llvm/trunk/lib/Fuzzer/<wbr class="">CMakeLists.txt (original)<br class="">> +++ llvm/trunk/lib/Fuzzer/<wbr class="">CMakeLists.txt Fri Aug 11 16:03:22 2017<br class="">> @@ -31,6 +31,7 @@ endif()<br class="">><br class="">> if (LIBFUZZER_ENABLE)<br class="">> add_library(<wbr class="">LLVMFuzzerNoMainObjects OBJECT<br class="">> + FuzzerClangCounters.cpp<br class="">> FuzzerCrossOver.cpp<br class="">> FuzzerDriver.cpp<br class="">> FuzzerExtFunctionsDlsym.cpp<br class="">><br class="">> Added: llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerClangCounters.cpp<br class="">> URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerClangCounters.cpp?rev=310771&view=auto" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-<wbr class="">project/llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerClangCounters.cpp?rev=<wbr class="">310771&view=auto</a><br class="">> ==============================<wbr class="">==============================<wbr class="">==================<br class="">> --- llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerClangCounters.cpp (added)<br class="">> +++ llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerClangCounters.cpp Fri Aug 11 16:03:22 2017<br class="">> @@ -0,0 +1,49 @@<br class="">> +//===- FuzzerExtraCounters.cpp - Extra coverage counters ------------------===//<br class="">> +//<br class="">> +// The LLVM Compiler Infrastructure<br class="">> +//<br class="">> +// This file is distributed under the University of Illinois Open Source<br class="">> +// License. See LICENSE.TXT for details.<br class="">> +//<br class="">> +//===------------------------<wbr class="">------------------------------<wbr class="">----------------===//<br class="">> +// Coverage counters from Clang's SourceBasedCodeCoverage.<br class="">> +//===------------------------<wbr class="">------------------------------<wbr class="">----------------===//<br class="">> +<br class="">> +// Support for SourceBasedCodeCoverage is experimental:<br class="">> +// * Works only for the main binary, not DSOs yet.<br class="">> +// * Works only on Linux.<br class="">> +// * Does not implement print_pcs/print_coverage yet.<br class="">> +// * Is not fully evaluated for performance and sensitivity.<br class="">> +// We expect large performance drop due to 64-bit counters,<br class="">> +// and *maybe* better sensitivity due to more fine-grained counters.<br class="">> +// Preliminary comparison on a single benchmark (RE2) shows<br class="">> +// a bit worse sensitivity though.<br class="">> +<br class="">> +#include "FuzzerDefs.h"<br class="">> +<br class="">> +#if LIBFUZZER_LINUX<br class="">> +__attribute__((weak)) extern uint64_t __start___llvm_prf_cnts;<br class="">> +__attribute__((weak)) extern uint64_t __stop___llvm_prf_cnts;<br class="">> +namespace fuzzer {<br class="">> +uint64_t *ClangCountersBegin() { return &__start___llvm_prf_cnts; }<br class="">> +uint64_t *ClangCountersEnd() { return &__stop___llvm_prf_cnts; }<br class="">> +} // namespace fuzzer<br class="">> +#else<br class="">> +// TODO: Implement on Mac (if the data shows it's worth it).<br class="">> +//__attribute__((visibility("<wbr class="">hidden")))<br class="">> +//extern uint64_t CountersStart __asm("section$start$__DATA$__<wbr class="">llvm_prf_cnts");<br class="">> +//__attribute__((visibility("<wbr class="">hidden")))<br class="">> +//extern uint64_t CountersEnd __asm("section$end$__DATA$__<wbr class="">llvm_prf_cnts");<br class="">> +namespace fuzzer {<br class="">> +uint64_t *ClangCountersBegin() { return nullptr; }<br class=""><br class=""></div></div>Use: uint64_t *__llvm_profile_begin_<wbr class="">counters(void), and *_end_counters()?<br class=""><span class="gmail-"><br class="">> +uint64_t *ClangCountersEnd() { return nullptr; }<br class="">> +} // namespace fuzzer<br class="">> +#endif<br class="">> +<br class="">> +namespace fuzzer {<br class="">> +ATTRIBUTE_NO_SANITIZE_ALL<br class="">> +void ClearClangCounters() { // hand-written memset, don't asan-ify.<br class="">> + for (auto P = ClangCountersBegin(); P < ClangCountersEnd(); P++)<br class="">> + *P = 0;<br class="">> +}<br class=""><br class=""></span>Use: void __llvm_profile_reset_counters(<wbr class="">void); ?<br class=""><div class=""><div class="gmail-h5"><br class="">> +}<br class="">><br class="">> Modified: llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerDefs.h<br class="">> URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerDefs.h?rev=310771&r1=310770&r2=310771&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-<wbr class="">project/llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerDefs.h?rev=310771&r1=<wbr class="">310770&r2=310771&view=diff</a><br class="">> ==============================<wbr class="">==============================<wbr class="">==================<br class="">> --- llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerDefs.h (original)<br class="">> +++ llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerDefs.h Fri Aug 11 16:03:22 2017<br class="">> @@ -123,6 +123,10 @@ uint8_t *ExtraCountersBegin();<br class="">> uint8_t *ExtraCountersEnd();<br class="">> void ClearExtraCounters();<br class="">><br class="">> +uint64_t *ClangCountersBegin();<br class="">> +uint64_t *ClangCountersEnd();<br class="">> +void ClearClangCounters();<br class="">> +<br class="">> } // namespace fuzzer<br class="">><br class="">> #endif // LLVM_FUZZER_DEFS_H<br class="">><br class="">> Modified: llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.cpp<br class="">> URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerTracePC.cpp?rev=310771&r1=310770&r2=310771&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-<wbr class="">project/llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.cpp?rev=310771&<wbr class="">r1=310770&r2=310771&view=diff</a><br class="">> ==============================<wbr class="">==============================<wbr class="">==================<br class="">> --- llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.cpp (original)<br class="">> +++ llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.cpp Fri Aug 11 16:03:22 2017<br class="">> @@ -126,6 +126,8 @@ void TracePC::PrintModuleInfo() {<br class="">> _Exit(1);<br class="">> }<br class="">> }<br class="">> + if (size_t NumClangCounters = ClangCountersEnd() - ClangCountersBegin())<br class="">> + Printf("INFO: %zd Clang Coverage Counters\n", NumClangCounters);<br class="">> }<br class="">><br class="">> ATTRIBUTE_NO_SANITIZE_ALL<br class="">> @@ -137,13 +139,12 @@ void TracePC::HandleCallerCallee(<wbr class="">uintptr<br class="">> }<br class="">><br class="">> void TracePC::UpdateObservedPCs() {<br class=""><br class=""></div></div>I'm a bit confused by what this is doing. Why is it interesting to track the indices of covered counters?</blockquote><div class=""><br class=""></div><div class="">This is just for printing statistics: how many different PCs are covered in total during the run. </div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">Isn't this something the recorded profile can tell you?<br class=""></blockquote><div class=""><br class=""></div><div class="">What's "recorded profile"? </div><div class="">Given that I flush the array of counters after every input, there is no "recorded profile".</div></div></div></blockquote><div><br class=""></div>I see, there's no profile written out to disk here.</div><div><br class=""></div><div>vedant</div><div><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class=""> </div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><br class="">best,<br class="">vedant<br class=""><div class="gmail-HOEnZb"><div class="gmail-h5"><br class="">> + auto Observe = [&](uintptr_t PC) {<br class="">> + bool Inserted = ObservedPCs.insert(PC).second;<br class="">> + if (Inserted && DoPrintNewPCs)<br class="">> + PrintPC("\tNEW_PC: %p %F %L\n", "\tNEW_PC: %p\n", PC + 1);<br class="">> + };<br class="">> if (NumPCsInPCTables) {<br class="">> - auto Observe = [&](uintptr_t PC) {<br class="">> - bool Inserted = ObservedPCs.insert(PC).second;<br class="">> - if (Inserted && DoPrintNewPCs)<br class="">> - PrintPC("\tNEW_PC: %p %F %L\n", "\tNEW_PC: %p\n", PC + 1);<br class="">> - };<br class="">> -<br class="">> if (NumInline8bitCounters == NumPCsInPCTables) {<br class="">> for (size_t i = 0; i < NumModulesWithInline8bitCounte<wbr class="">rs; i++) {<br class="">> uint8_t *Beg = ModuleCounters[i].Start;<br class="">> @@ -167,6 +168,13 @@ void TracePC::UpdateObservedPCs() {<br class="">> }<br class="">> }<br class="">> }<br class="">> + if (size_t NumClangCounters =<br class="">> + ClangCountersEnd() - ClangCountersBegin()) {<br class="">> + auto P = ClangCountersBegin();<br class="">> + for (size_t Idx = 0; Idx < NumClangCounters; Idx++)<br class="">> + if (P[Idx])<br class="">> + Observe((uintptr_t)Idx);<br class="">> + }<br class="">> }<br class="">><br class="">> inline ALWAYS_INLINE uintptr_t GetPreviousInstructionPc(<wbr class="">uintptr_t PC) {<br class="">><br class="">> Modified: llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.h<br class="">> URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerTracePC.h?rev=310771&r1=310770&r2=310771&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-<wbr class="">project/llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.h?rev=310771&r1=<wbr class="">310770&r2=310771&view=diff</a><br class="">> ==============================<wbr class="">==============================<wbr class="">==================<br class="">> --- llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.h (original)<br class="">> +++ llvm/trunk/lib/Fuzzer/<wbr class="">FuzzerTracePC.h Fri Aug 11 16:03:22 2017<br class="">> @@ -91,6 +91,7 @@ class TracePC {<br class="">> memset(Counters(), 0, GetNumPCs());<br class="">> ClearExtraCounters();<br class="">> ClearInlineCounters();<br class="">> + ClearClangCounters();<br class="">> }<br class="">><br class="">> void ClearInlineCounters();<br class="">> @@ -196,14 +197,9 @@ void ForEachNonZeroByte(const uint8_t *B<br class="">> Handle8bitCounter(<wbr class="">FirstFeature, P - Begin, V);<br class="">> }<br class="">><br class="">> -template <class Callback> // bool Callback(size_t Feature)<br class="">> -ATTRIBUTE_NO_SANITIZE_ADDRESS<br class="">> -__attribute__((noinline))<br class="">> -void TracePC::CollectFeatures(<wbr class="">Callback HandleFeature) const {<br class="">> - uint8_t *Counters = this->Counters();<br class="">> - size_t N = GetNumPCs();<br class="">> - auto Handle8bitCounter = [&](size_t FirstFeature,<br class="">> - size_t Idx, uint8_t Counter) {<br class="">> +// Given a non-zero Counters returns a number in [0,7].<br class="">> +template<class T><br class="">> +unsigned CounterToFeature(T Counter) {<br class="">> assert(Counter);<br class="">> unsigned Bit = 0;<br class="">> /**/ if (Counter >= 128) Bit = 7;<br class="">> @@ -213,7 +209,18 @@ void TracePC::CollectFeatures(<wbr class="">Callback H<br class="">> else if (Counter >= 4) Bit = 3;<br class="">> else if (Counter >= 3) Bit = 2;<br class="">> else if (Counter >= 2) Bit = 1;<br class="">> - HandleFeature(FirstFeature + Idx * 8 + Bit);<br class="">> + return Bit;<br class="">> +}<br class="">> +<br class="">> +template <class Callback> // bool Callback(size_t Feature)<br class="">> +ATTRIBUTE_NO_SANITIZE_ADDRESS<br class="">> +__attribute__((noinline))<br class="">> +void TracePC::CollectFeatures(<wbr class="">Callback HandleFeature) const {<br class="">> + uint8_t *Counters = this->Counters();<br class="">> + size_t N = GetNumPCs();<br class="">> + auto Handle8bitCounter = [&](size_t FirstFeature,<br class="">> + size_t Idx, uint8_t Counter) {<br class="">> + HandleFeature(FirstFeature + Idx * 8 + CounterToFeature(Counter));<br class="">> };<br class="">><br class="">> size_t FirstFeature = 0;<br class="">> @@ -231,6 +238,14 @@ void TracePC::CollectFeatures(<wbr class="">Callback H<br class="">> }<br class="">> }<br class="">><br class="">> + if (size_t NumClangCounters = ClangCountersEnd() - ClangCountersBegin()) {<br class="">> + auto P = ClangCountersBegin();<br class="">> + for (size_t Idx = 0; Idx < NumClangCounters; Idx++)<br class="">> + if (auto Cnt = P[Idx])<br class="">> + HandleFeature(FirstFeature + Idx * 8 + CounterToFeature(Cnt));<br class="">> + FirstFeature += NumClangCounters;<br class="">> + }<br class="">> +<br class="">> ForEachNonZeroByte(<wbr class="">ExtraCountersBegin(), ExtraCountersEnd(), FirstFeature,<br class="">> Handle8bitCounter);<br class="">> FirstFeature += (ExtraCountersEnd() - ExtraCountersBegin()) * 8;<br class="">><br class="">> Added: llvm/trunk/lib/Fuzzer/test/<wbr class="">fprofile-instr-generate.test<br class="">> URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/fprofile-instr-generate.test?rev=310771&view=auto" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-<wbr class="">project/llvm/trunk/lib/Fuzzer/<wbr class="">test/fprofile-instr-generate.<wbr class="">test?rev=310771&view=auto</a><br class="">> ==============================<wbr class="">==============================<wbr class="">==================<br class="">> --- llvm/trunk/lib/Fuzzer/test/<wbr class="">fprofile-instr-generate.test (added)<br class="">> +++ llvm/trunk/lib/Fuzzer/test/<wbr class="">fprofile-instr-generate.test Fri Aug 11 16:03:22 2017<br class="">> @@ -0,0 +1,7 @@<br class="">> +# Test libFuzzer + -fprofile-instr-generate<br class="">> +REQUIRES: linux<br class="">> +RUN: %cpp_compiler %S/SimpleTest.cpp -fsanitize-coverage=0 -fprofile-instr-generate -o %t-SimpleTest-fprofile-instr-<wbr class="">generate<br class="">> +CHECK-NOT: INFO: Loaded 1 modules<br class="">> +CHECK: INFO: {{.*}} Clang Coverage Counters<br class="">> +CHECK: BINGO<br class="">> +RUN: not %t-SimpleTest-fprofile-instr-<wbr class="">generate -runs=1000000 -seed=1 2>&1 | FileCheck %s<br class="">><br class="">><br class="">> ______________________________<wbr class="">_________________<br class="">> llvm-commits mailing list<br class="">><span class="Apple-converted-space"> </span><a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a><br class="">><span class="Apple-converted-space"> </span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/<wbr class="">mailman/listinfo/llvm-commits</a></div></div></blockquote></div></div></blockquote></div><br class=""></body></html>