[llvm] r309646 - [libFuzzer] enable -fsanitize-coverage=pc-table for all tests
Alex L via llvm-commits
llvm-commits at lists.llvm.org
Tue Aug 8 02:12:45 PDT 2017
Thanks for the fix and the detailed explanation!
We are using macOS 10.11.6 on the bot.
Alex
On 4 August 2017 at 19:45, Kostya Serebryany <kcc at google.com> wrote:
> Hi Alex,
> Just curious, what is the OS on this bot?
>
> Anyway...
>
> libFuzzer implements a random search algorithm and so the tests are
> inherently random.
> To partially fix this we run most tests with a fixed RNG seed (e.g.
> -seed=1) but this only makes
> them deterministic on one given platform (i.e. one given implementation of
> C++'s std::mt19937).
>
> This means that the test may pass on my platform (always) and fail on
> yours (always).
>
> For this particular test, I've run it 1000 with different seeds and it
> failed 53 times.
> However, if I increase the number of iterations for this test from 100,000
> to 1,000,000 the test passes 1000 out of 1000.
>
> Hence I hope r310091 will fix your bot.
>
> Thanks for the report!
>
> (BTW, check-fuzzer is broken currently for an unrelated reason: r310077)
>
> --kcc
>
> On Fri, Aug 4, 2017 at 5:41 AM, Alex L <arphaman at gmail.com> wrote:
>
>> Hi Kostya,
>>
>> This change seems to have broken our Fuzzer buildbot:
>> http://lab.llvm.org:8080/green/job/libFuzzer/
>>
>> Now trace-pc.test is failing (most of the time, it seems to have
>> succeeded once or twice). I've managed to reproduce it on the bot itself,
>> but not on my local machine. I've attached the output of
>> LLVMFuzzer-SimpleTest-TracePC on the bot in a file. It doesn't contain
>> BINGO, so the test fails. Do you know what is going on here? I've confirmed
>> that reverting this commit fixes the issue.
>>
>> Let me know if there's anything else I can do,
>> Alex
>>
>>
>>
>>
>> On 1 August 2017 at 19:04, Kostya Serebryany via llvm-commits <
>> llvm-commits at lists.llvm.org> wrote:
>>
>>> r309716 should make the windows bot green.
>>> Redi, Zack, please help me fix this on windows ASAP, this is blocking my
>>> progress (should be simple).
>>>
>>> On Tue, Aug 1, 2017 at 10:56 AM, Kostya Serebryany <kcc at google.com>
>>> wrote:
>>>
>>>> Hold on, I'll revert only the part that actually causes the failure.
>>>>
>>>>
>>>> On Tue, Aug 1, 2017 at 10:46 AM, Vitaly Buka <vitalybuka at google.com>
>>>> wrote:
>>>>
>>>>> I am going revert this change?
>>>>>
>>>>> On Tue, Aug 1, 2017 at 8:16 AM, Kostya Serebryany <kcc at google.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Reid, Zach,
>>>>>>
>>>>>> I've introduced yet another kind of coverage instrumentation, which
>>>>>> doesn't work on windows out of the box.
>>>>>> Could you please help?
>>>>>> This is again something about section naming.
>>>>>>
>>>>>> --kcc
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 1, 2017 at 1:27 AM, Vitaly Buka <vitalybuka at google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Broken by this patch
>>>>>>> http://lab.llvm.org:8011/builders/sanitizer-windows/builds/14792
>>>>>>>
>>>>>>> On Mon, Jul 31, 2017 at 5:48 PM, Kostya Serebryany via llvm-commits
>>>>>>> <llvm-commits at lists.llvm.org> wrote:
>>>>>>>
>>>>>>>> Author: kcc
>>>>>>>> Date: Mon Jul 31 17:48:44 2017
>>>>>>>> New Revision: 309646
>>>>>>>>
>>>>>>>> URL: http://llvm.org/viewvc/llvm-project?rev=309646&view=rev
>>>>>>>> Log:
>>>>>>>> [libFuzzer] enable -fsanitize-coverage=pc-table for all tests
>>>>>>>>
>>>>>>>> Modified:
>>>>>>>> llvm/trunk/lib/Fuzzer/FuzzerTracePC.cpp
>>>>>>>> llvm/trunk/lib/Fuzzer/FuzzerTracePC.h
>>>>>>>> llvm/trunk/lib/Fuzzer/test/CMakeLists.txt
>>>>>>>> llvm/trunk/lib/Fuzzer/test/inline-8bit-counters.test
>>>>>>>>
>>>>>>>> Modified: llvm/trunk/lib/Fuzzer/FuzzerTracePC.cpp
>>>>>>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/Fu
>>>>>>>> zzerTracePC.cpp?rev=309646&r1=309645&r2=309646&view=diff
>>>>>>>> ============================================================
>>>>>>>> ==================
>>>>>>>> --- llvm/trunk/lib/Fuzzer/FuzzerTracePC.cpp (original)
>>>>>>>> +++ llvm/trunk/lib/Fuzzer/FuzzerTracePC.cpp Mon Jul 31 17:48:44
>>>>>>>> 2017
>>>>>>>> @@ -72,7 +72,7 @@ void TracePC::HandlePCsInit(const uint8_
>>>>>>>> if (NumPCTables && ModulePCTable[NumPCTables - 1].Start == B)
>>>>>>>> return;
>>>>>>>> assert(NumPCTables < sizeof(ModulePCTable) /
>>>>>>>> sizeof(ModulePCTable[0]));
>>>>>>>> ModulePCTable[NumPCTables++] = {B, E};
>>>>>>>> - NumPCsInPCTables = E - B;
>>>>>>>> + NumPCsInPCTables += E - B;
>>>>>>>> }
>>>>>>>>
>>>>>>>> void TracePC::HandleInit(uint32_t *Start, uint32_t *Stop) {
>>>>>>>> @@ -95,26 +95,36 @@ void TracePC::HandleInit(uint32_t *Start
>>>>>>>>
>>>>>>>> void TracePC::PrintModuleInfo() {
>>>>>>>> if (NumGuards) {
>>>>>>>> - Printf("INFO: Loaded %zd modules (%zd guards): ", NumModules,
>>>>>>>> NumGuards);
>>>>>>>> + Printf("INFO: Loaded %zd modules (%zd guards): ",
>>>>>>>> NumModules, NumGuards);
>>>>>>>> for (size_t i = 0; i < NumModules; i++)
>>>>>>>> - Printf("[%p, %p), ", Modules[i].Start, Modules[i].Stop);
>>>>>>>> + Printf("%zd [%p, %p), ", Modules[i].Stop - Modules[i].Start,
>>>>>>>> + Modules[i].Start, Modules[i].Stop);
>>>>>>>> Printf("\n");
>>>>>>>> }
>>>>>>>> if (NumModulesWithInline8bitCounters) {
>>>>>>>> - Printf("INFO: Loaded %zd modules with %zd inline 8-bit
>>>>>>>> counters: ",
>>>>>>>> + Printf("INFO: Loaded %zd modules (%zd inline 8-bit
>>>>>>>> counters): ",
>>>>>>>> NumModulesWithInline8bitCounters,
>>>>>>>> NumInline8bitCounters);
>>>>>>>> for (size_t i = 0; i < NumModulesWithInline8bitCounters; i++)
>>>>>>>> - Printf("[%p, %p), ", ModuleCounters[i].Start,
>>>>>>>> ModuleCounters[i].Stop);
>>>>>>>> + Printf("%zd [%p, %p), ", ModuleCounters[i].Stop -
>>>>>>>> ModuleCounters[i].Start,
>>>>>>>> + ModuleCounters[i].Start, ModuleCounters[i].Stop);
>>>>>>>> Printf("\n");
>>>>>>>> }
>>>>>>>> if (NumPCTables) {
>>>>>>>> - Printf("INFO: Loaded %zd PC tables, %zd PCs: ", NumPCTables,
>>>>>>>> + Printf("INFO: Loaded %zd PC tables (%zd PCs): ", NumPCTables,
>>>>>>>> NumPCsInPCTables);
>>>>>>>> for (size_t i = 0; i < NumPCTables; i++) {
>>>>>>>> - Printf("[%p,%p), ", ModulePCTable[i].Start,
>>>>>>>> ModulePCTable[i].Stop,
>>>>>>>> - ModulePCTable[i].Stop - ModulePCTable[i].Start);
>>>>>>>> + Printf("%zd [%p,%p), ", ModulePCTable[i].Stop -
>>>>>>>> ModulePCTable[i].Start,
>>>>>>>> + ModulePCTable[i].Start, ModulePCTable[i].Stop);
>>>>>>>> }
>>>>>>>> Printf("\n");
>>>>>>>> +
>>>>>>>> + if ((NumGuards && NumGuards != NumPCsInPCTables) ||
>>>>>>>> + (NumInline8bitCounters && NumInline8bitCounters !=
>>>>>>>> NumPCsInPCTables)) {
>>>>>>>> + Printf("ERROR: The size of coverage PC tables does not match
>>>>>>>> the"
>>>>>>>> + " number of instrumented PCs. This might be a bug in
>>>>>>>> the compiler,"
>>>>>>>> + " please contact the libFuzzer developers.\n");
>>>>>>>> + _Exit(1);
>>>>>>>> + }
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> Modified: llvm/trunk/lib/Fuzzer/FuzzerTracePC.h
>>>>>>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/Fu
>>>>>>>> zzerTracePC.h?rev=309646&r1=309645&r2=309646&view=diff
>>>>>>>> ============================================================
>>>>>>>> ==================
>>>>>>>> --- llvm/trunk/lib/Fuzzer/FuzzerTracePC.h (original)
>>>>>>>> +++ llvm/trunk/lib/Fuzzer/FuzzerTracePC.h Mon Jul 31 17:48:44 2017
>>>>>>>> @@ -86,7 +86,8 @@ class TracePC {
>>>>>>>>
>>>>>>>> void ResetMaps() {
>>>>>>>> ValueProfileMap.Reset();
>>>>>>>> - memset(Counters(), 0, GetNumPCs());
>>>>>>>> + if (NumModules)
>>>>>>>> + memset(Counters(), 0, GetNumPCs());
>>>>>>>> ClearExtraCounters();
>>>>>>>> ClearInlineCounters();
>>>>>>>> }
>>>>>>>>
>>>>>>>> Modified: llvm/trunk/lib/Fuzzer/test/CMakeLists.txt
>>>>>>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/te
>>>>>>>> st/CMakeLists.txt?rev=309646&r1=309645&r2=309646&view=diff
>>>>>>>> ============================================================
>>>>>>>> ==================
>>>>>>>> --- llvm/trunk/lib/Fuzzer/test/CMakeLists.txt (original)
>>>>>>>> +++ llvm/trunk/lib/Fuzzer/test/CMakeLists.txt Mon Jul 31 17:48:44
>>>>>>>> 2017
>>>>>>>> @@ -15,7 +15,7 @@ foreach (VARNAME ${variables_to_filter})
>>>>>>>> endforeach()
>>>>>>>>
>>>>>>>> # Enable the coverage instrumentation (it is disabled for the
>>>>>>>> Fuzzer lib).
>>>>>>>> -set(CMAKE_CXX_FLAGS "${LIBFUZZER_FLAGS_BASE}
>>>>>>>> -fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp,trace-div,trace-gep
>>>>>>>> -gline-tables-only")
>>>>>>>> +set(CMAKE_CXX_FLAGS "${LIBFUZZER_FLAGS_BASE}
>>>>>>>> -fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp,trace-div,trace-gep,pc-table
>>>>>>>> -gline-tables-only")
>>>>>>>>
>>>>>>>> if(MSVC)
>>>>>>>> # For tests use the CRT specified for release build
>>>>>>>>
>>>>>>>> Modified: llvm/trunk/lib/Fuzzer/test/inline-8bit-counters.test
>>>>>>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/te
>>>>>>>> st/inline-8bit-counters.test?rev=309646&r1=309645&r2=309646&
>>>>>>>> view=diff
>>>>>>>> ============================================================
>>>>>>>> ==================
>>>>>>>> --- llvm/trunk/lib/Fuzzer/test/inline-8bit-counters.test (original)
>>>>>>>> +++ llvm/trunk/lib/Fuzzer/test/inline-8bit-counters.test Mon Jul
>>>>>>>> 31 17:48:44 2017
>>>>>>>> @@ -1,4 +1,4 @@
>>>>>>>> REQUIRES: linux
>>>>>>>> -CHECK: INFO: Loaded 1 modules with {{.*}} inline 8-bit counters
>>>>>>>> +CHECK: INFO: Loaded 1 modules ({{.*}} inline 8-bit counters)
>>>>>>>> CHECK: BINGO
>>>>>>>> RUN: not LLVMFuzzer-SimpleTest-Inline8bitCounters -runs=1000000
>>>>>>>> -seed=1 2>&1 | FileCheck %s
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> llvm-commits mailing list
>>>>>>>> llvm-commits at lists.llvm.org
>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170808/43c069b2/attachment.html>
More information about the llvm-commits
mailing list