[llvm] r309646 - [libFuzzer] enable -fsanitize-coverage=pc-table for all tests
Kostya Serebryany via llvm-commits
llvm-commits at lists.llvm.org
Fri Aug 4 11:45:20 PDT 2017
Hi Alex,
Just curious, what is the OS on this bot?
Anyway...
libFuzzer implements a random search algorithm and so the tests are
inherently random.
To partially fix this we run most tests with a fixed RNG seed (e.g.
-seed=1) but this only makes
them deterministic on one given platform (i.e. one given implementation of
C++'s std::mt19937).
This means that the test may pass on my platform (always) and fail on yours
(always).
For this particular test, I've run it 1000 with different seeds and it
failed 53 times.
However, if I increase the number of iterations for this test from 100,000
to 1,000,000 the test passes 1000 out of 1000.
Hence I hope r310091 will fix your bot.
Thanks for the report!
(BTW, check-fuzzer is broken currently for an unrelated reason: r310077)
--kcc
On Fri, Aug 4, 2017 at 5:41 AM, Alex L <arphaman at gmail.com> wrote:
> Hi Kostya,
>
> This change seems to have broken our Fuzzer buildbot:
> http://lab.llvm.org:8080/green/job/libFuzzer/
>
> Now trace-pc.test is failing (most of the time, it seems to have succeeded
> once or twice). I've managed to reproduce it on the bot itself, but not on
> my local machine. I've attached the output of LLVMFuzzer-SimpleTest-TracePC
> on the bot in a file. It doesn't contain BINGO, so the test fails. Do you
> know what is going on here? I've confirmed that reverting this commit fixes
> the issue.
>
> Let me know if there's anything else I can do,
> Alex
>
>
>
>
> On 1 August 2017 at 19:04, Kostya Serebryany via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> r309716 should make the windows bot green.
>> Redi, Zack, please help me fix this on windows ASAP, this is blocking my
>> progress (should be simple).
>>
>> On Tue, Aug 1, 2017 at 10:56 AM, Kostya Serebryany <kcc at google.com>
>> wrote:
>>
>>> Hold on, I'll revert only the part that actually causes the failure.
>>>
>>>
>>> On Tue, Aug 1, 2017 at 10:46 AM, Vitaly Buka <vitalybuka at google.com>
>>> wrote:
>>>
>>>> I am going revert this change?
>>>>
>>>> On Tue, Aug 1, 2017 at 8:16 AM, Kostya Serebryany <kcc at google.com>
>>>> wrote:
>>>>
>>>>> Hi Reid, Zach,
>>>>>
>>>>> I've introduced yet another kind of coverage instrumentation, which
>>>>> doesn't work on windows out of the box.
>>>>> Could you please help?
>>>>> This is again something about section naming.
>>>>>
>>>>> --kcc
>>>>>
>>>>>
>>>>> On Tue, Aug 1, 2017 at 1:27 AM, Vitaly Buka <vitalybuka at google.com>
>>>>> wrote:
>>>>>
>>>>>> Broken by this patch
>>>>>> http://lab.llvm.org:8011/builders/sanitizer-windows/builds/14792
>>>>>>
>>>>>> On Mon, Jul 31, 2017 at 5:48 PM, Kostya Serebryany via llvm-commits <
>>>>>> llvm-commits at lists.llvm.org> wrote:
>>>>>>
>>>>>>> Author: kcc
>>>>>>> Date: Mon Jul 31 17:48:44 2017
>>>>>>> New Revision: 309646
>>>>>>>
>>>>>>> URL: http://llvm.org/viewvc/llvm-project?rev=309646&view=rev
>>>>>>> Log:
>>>>>>> [libFuzzer] enable -fsanitize-coverage=pc-table for all tests
>>>>>>>
>>>>>>> Modified:
>>>>>>> llvm/trunk/lib/Fuzzer/FuzzerTracePC.cpp
>>>>>>> llvm/trunk/lib/Fuzzer/FuzzerTracePC.h
>>>>>>> llvm/trunk/lib/Fuzzer/test/CMakeLists.txt
>>>>>>> llvm/trunk/lib/Fuzzer/test/inline-8bit-counters.test
>>>>>>>
>>>>>>> Modified: llvm/trunk/lib/Fuzzer/FuzzerTracePC.cpp
>>>>>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/Fu
>>>>>>> zzerTracePC.cpp?rev=309646&r1=309645&r2=309646&view=diff
>>>>>>> ============================================================
>>>>>>> ==================
>>>>>>> --- llvm/trunk/lib/Fuzzer/FuzzerTracePC.cpp (original)
>>>>>>> +++ llvm/trunk/lib/Fuzzer/FuzzerTracePC.cpp Mon Jul 31 17:48:44 2017
>>>>>>> @@ -72,7 +72,7 @@ void TracePC::HandlePCsInit(const uint8_
>>>>>>> if (NumPCTables && ModulePCTable[NumPCTables - 1].Start == B)
>>>>>>> return;
>>>>>>> assert(NumPCTables < sizeof(ModulePCTable) /
>>>>>>> sizeof(ModulePCTable[0]));
>>>>>>> ModulePCTable[NumPCTables++] = {B, E};
>>>>>>> - NumPCsInPCTables = E - B;
>>>>>>> + NumPCsInPCTables += E - B;
>>>>>>> }
>>>>>>>
>>>>>>> void TracePC::HandleInit(uint32_t *Start, uint32_t *Stop) {
>>>>>>> @@ -95,26 +95,36 @@ void TracePC::HandleInit(uint32_t *Start
>>>>>>>
>>>>>>> void TracePC::PrintModuleInfo() {
>>>>>>> if (NumGuards) {
>>>>>>> - Printf("INFO: Loaded %zd modules (%zd guards): ", NumModules,
>>>>>>> NumGuards);
>>>>>>> + Printf("INFO: Loaded %zd modules (%zd guards): ", NumModules,
>>>>>>> NumGuards);
>>>>>>> for (size_t i = 0; i < NumModules; i++)
>>>>>>> - Printf("[%p, %p), ", Modules[i].Start, Modules[i].Stop);
>>>>>>> + Printf("%zd [%p, %p), ", Modules[i].Stop - Modules[i].Start,
>>>>>>> + Modules[i].Start, Modules[i].Stop);
>>>>>>> Printf("\n");
>>>>>>> }
>>>>>>> if (NumModulesWithInline8bitCounters) {
>>>>>>> - Printf("INFO: Loaded %zd modules with %zd inline 8-bit
>>>>>>> counters: ",
>>>>>>> + Printf("INFO: Loaded %zd modules (%zd inline 8-bit counters):
>>>>>>> ",
>>>>>>> NumModulesWithInline8bitCounters,
>>>>>>> NumInline8bitCounters);
>>>>>>> for (size_t i = 0; i < NumModulesWithInline8bitCounters; i++)
>>>>>>> - Printf("[%p, %p), ", ModuleCounters[i].Start,
>>>>>>> ModuleCounters[i].Stop);
>>>>>>> + Printf("%zd [%p, %p), ", ModuleCounters[i].Stop -
>>>>>>> ModuleCounters[i].Start,
>>>>>>> + ModuleCounters[i].Start, ModuleCounters[i].Stop);
>>>>>>> Printf("\n");
>>>>>>> }
>>>>>>> if (NumPCTables) {
>>>>>>> - Printf("INFO: Loaded %zd PC tables, %zd PCs: ", NumPCTables,
>>>>>>> + Printf("INFO: Loaded %zd PC tables (%zd PCs): ", NumPCTables,
>>>>>>> NumPCsInPCTables);
>>>>>>> for (size_t i = 0; i < NumPCTables; i++) {
>>>>>>> - Printf("[%p,%p), ", ModulePCTable[i].Start,
>>>>>>> ModulePCTable[i].Stop,
>>>>>>> - ModulePCTable[i].Stop - ModulePCTable[i].Start);
>>>>>>> + Printf("%zd [%p,%p), ", ModulePCTable[i].Stop -
>>>>>>> ModulePCTable[i].Start,
>>>>>>> + ModulePCTable[i].Start, ModulePCTable[i].Stop);
>>>>>>> }
>>>>>>> Printf("\n");
>>>>>>> +
>>>>>>> + if ((NumGuards && NumGuards != NumPCsInPCTables) ||
>>>>>>> + (NumInline8bitCounters && NumInline8bitCounters !=
>>>>>>> NumPCsInPCTables)) {
>>>>>>> + Printf("ERROR: The size of coverage PC tables does not match
>>>>>>> the"
>>>>>>> + " number of instrumented PCs. This might be a bug in
>>>>>>> the compiler,"
>>>>>>> + " please contact the libFuzzer developers.\n");
>>>>>>> + _Exit(1);
>>>>>>> + }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> Modified: llvm/trunk/lib/Fuzzer/FuzzerTracePC.h
>>>>>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/Fu
>>>>>>> zzerTracePC.h?rev=309646&r1=309645&r2=309646&view=diff
>>>>>>> ============================================================
>>>>>>> ==================
>>>>>>> --- llvm/trunk/lib/Fuzzer/FuzzerTracePC.h (original)
>>>>>>> +++ llvm/trunk/lib/Fuzzer/FuzzerTracePC.h Mon Jul 31 17:48:44 2017
>>>>>>> @@ -86,7 +86,8 @@ class TracePC {
>>>>>>>
>>>>>>> void ResetMaps() {
>>>>>>> ValueProfileMap.Reset();
>>>>>>> - memset(Counters(), 0, GetNumPCs());
>>>>>>> + if (NumModules)
>>>>>>> + memset(Counters(), 0, GetNumPCs());
>>>>>>> ClearExtraCounters();
>>>>>>> ClearInlineCounters();
>>>>>>> }
>>>>>>>
>>>>>>> Modified: llvm/trunk/lib/Fuzzer/test/CMakeLists.txt
>>>>>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/te
>>>>>>> st/CMakeLists.txt?rev=309646&r1=309645&r2=309646&view=diff
>>>>>>> ============================================================
>>>>>>> ==================
>>>>>>> --- llvm/trunk/lib/Fuzzer/test/CMakeLists.txt (original)
>>>>>>> +++ llvm/trunk/lib/Fuzzer/test/CMakeLists.txt Mon Jul 31 17:48:44
>>>>>>> 2017
>>>>>>> @@ -15,7 +15,7 @@ foreach (VARNAME ${variables_to_filter})
>>>>>>> endforeach()
>>>>>>>
>>>>>>> # Enable the coverage instrumentation (it is disabled for the
>>>>>>> Fuzzer lib).
>>>>>>> -set(CMAKE_CXX_FLAGS "${LIBFUZZER_FLAGS_BASE}
>>>>>>> -fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp,trace-div,trace-gep
>>>>>>> -gline-tables-only")
>>>>>>> +set(CMAKE_CXX_FLAGS "${LIBFUZZER_FLAGS_BASE}
>>>>>>> -fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp,trace-div,trace-gep,pc-table
>>>>>>> -gline-tables-only")
>>>>>>>
>>>>>>> if(MSVC)
>>>>>>> # For tests use the CRT specified for release build
>>>>>>>
>>>>>>> Modified: llvm/trunk/lib/Fuzzer/test/inline-8bit-counters.test
>>>>>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/te
>>>>>>> st/inline-8bit-counters.test?rev=309646&r1=309645&r2=309646&
>>>>>>> view=diff
>>>>>>> ============================================================
>>>>>>> ==================
>>>>>>> --- llvm/trunk/lib/Fuzzer/test/inline-8bit-counters.test (original)
>>>>>>> +++ llvm/trunk/lib/Fuzzer/test/inline-8bit-counters.test Mon Jul 31
>>>>>>> 17:48:44 2017
>>>>>>> @@ -1,4 +1,4 @@
>>>>>>> REQUIRES: linux
>>>>>>> -CHECK: INFO: Loaded 1 modules with {{.*}} inline 8-bit counters
>>>>>>> +CHECK: INFO: Loaded 1 modules ({{.*}} inline 8-bit counters)
>>>>>>> CHECK: BINGO
>>>>>>> RUN: not LLVMFuzzer-SimpleTest-Inline8bitCounters -runs=1000000
>>>>>>> -seed=1 2>&1 | FileCheck %s
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> llvm-commits mailing list
>>>>>>> llvm-commits at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170804/687dcad5/attachment.html>
More information about the llvm-commits
mailing list