[llvm] r308069 - [libFuzzer] Allow non-fuzzer args after -ignore_remaining_args=1
Reid Kleckner via llvm-commits
llvm-commits at lists.llvm.org
Wed Jul 19 15:57:52 PDT 2017
I debugged this, and discovered something heinous. Basically, all the
LLVMFuzzer* hooks need to be marked export. On Linux, symbols appear in
shared objects by default (it's called default visibility). This isn't true
on Windows. Imagine that all objects are compiled with -fvisibility=hidden.
So, we need to export the symbols. How do the existing lib/Fuzzer tests do
this? In CMake! See test/CMakeLists.txt:
function(test_export_symbol target symbol)
if(MSVC)
set_target_properties(LLVMFuzzer-${target} PROPERTIES LINK_FLAGS
"-export:${symbol}")
endif()
endfunction()
test_export_symbol(InitializeTest "LLVMFuzzerInitialize")
test_export_symbol(BogusInitializeTest "LLVMFuzzerInitialize")
test_export_symbol(CustomCrossOverTest "LLVMFuzzerCustomCrossOver")
test_export_symbol(CustomMutatorTest "LLVMFuzzerCustomMutator")
I think we should move this to ifdefs in the test source. Sound good? I
will post a patch.
On Mon, Jul 17, 2017 at 5:58 PM, Kostya Serebryany <kcc at google.com> wrote:
> Let me add
> UNSUPPORTED: windows
>
> On Mon, Jul 17, 2017 at 5:50 PM, Reid Kleckner <rnk at google.com> wrote:
>
>> I didn't have time to get to this today. I can look at it tomorrow. In
>> the meantime, can somebody XFAIL this?
>>
>> On Mon, Jul 17, 2017 at 5:34 PM, Kostya Serebryany <kcc at google.com>
>> wrote:
>>
>>> +rnk
>>>
>>> On Mon, Jul 17, 2017 at 12:25 PM, Justin Bogner <mail at justinbogner.com>
>>> wrote:
>>>
>>>> The output on the bot isn't terribly informative and I don't have a
>>>> windows machine. Could somebody run the failing command for me and paste
>>>> the output? It's as follows:
>>>>
>>>> "LLVMFuzzer-FlagsTest" "--foo-bar" "-runs=10"
>>>> "-ignore_remaining_args=1" "--baz" "-help=1" "test"
>>>>
>>>> Kostya Serebryany <kcc at google.com> writes:
>>>> > This broke libFuzzer on Windows:
>>>> > http://lab.llvm.org:8011/builders/sanitizer-windows/builds/1
>>>> 4306/steps/run%20fuzzer%20tests/logs/stdio
>>>> > I may not have time to look at this today, sadly.
>>>> >
>>>> > FAIL: LLVMFuzzer :: fuzzer-flags.test (14 of 92)
>>>> > ******************** TEST 'LLVMFuzzer :: fuzzer-flags.test' FAILED
>>>> > ********************
>>>> > Script:
>>>> > --
>>>> > LLVMFuzzer-FlagsTest -foo_bar=1 2>&1 | FileCheck
>>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>>> > --check-prefix=FOO_BAR
>>>> > LLVMFuzzer-FlagsTest -runs=10 --max_len=100 2>&1 | FileCheck
>>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>>> > --check-prefix=DASH_DASH
>>>> > LLVMFuzzer-FlagsTest -help=1 2>&1 | FileCheck
>>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>>> > --check-prefix=NO_INTERNAL
>>>> > LLVMFuzzer-FlagsTest --foo-bar -runs=10 -ignore_remaining_args=1 --baz
>>>> > -help=1 test 2>&1 | FileCheck
>>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>>> > --check-prefix=PASSTHRU
>>>> > mkdir -p
>>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>>> tput\fuzzer-flags.test.tmp/T0
>>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>>> tput\fuzzer-flags.test.tmp/T1
>>>> > touch
>>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>>> tput\fuzzer-flags.test.tmp/T1/empty
>>>> > LLVMFuzzer-FlagsTest --foo-bar -merge=1
>>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>>> tput\fuzzer-flags.test.tmp/T0
>>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>>> tput\fuzzer-flags.test.tmp/T1
>>>> > -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck
>>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>>> > --check-prefix=PASSTHRU-MERGE
>>>> > --
>>>> > Exit Code: 1
>>>> >
>>>> > Command Output (stdout):
>>>> > --
>>>> > $ "LLVMFuzzer-FlagsTest" "-foo_bar=1"
>>>> > $ "FileCheck"
>>>> > "C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test"
>>>> > "--check-prefix=FOO_BAR"
>>>> > $ "LLVMFuzzer-FlagsTest" "-runs=10" "--max_len=100"
>>>> > $ "FileCheck"
>>>> > "C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test"
>>>> > "--check-prefix=DASH_DASH"
>>>> > $ "LLVMFuzzer-FlagsTest" "-help=1"
>>>> > $ "FileCheck"
>>>> > "C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test"
>>>> > "--check-prefix=NO_INTERNAL"
>>>> > $ "LLVMFuzzer-FlagsTest" "--foo-bar" "-runs=10"
>>>> > "-ignore_remaining_args=1" "--baz" "-help=1" "test"
>>>> > $ "FileCheck"
>>>> > "C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test"
>>>> > "--check-prefix=PASSTHRU"
>>>> > # command stderr:
>>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-fla
>>>> gs.test:13:11:
>>>> > error: expected string not found in input
>>>> >
>>>> > PASSTHRU: BINGO --foo-bar --baz -help=1 test
>>>> >
>>>> > ^
>>>> >
>>>> > <stdin>:1:1: note: scanning from here
>>>> >
>>>> > INFO: libFuzzer ignores flags that start with '--'
>>>> >
>>>> > ^
>>>> >
>>>> >
>>>> > error: command failed with exit status: 1
>>>> >
>>>> > --
>>>> >
>>>> > ********************
>>>> > Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
>>>> >
>>>> >
>>>> >
>>>> > On Fri, Jul 14, 2017 at 4:33 PM, Justin Bogner via llvm-commits <
>>>> > llvm-commits at lists.llvm.org> wrote:
>>>> >
>>>> >> Author: bogner
>>>> >> Date: Fri Jul 14 16:33:04 2017
>>>> >> New Revision: 308069
>>>> >>
>>>> >> URL: http://llvm.org/viewvc/llvm-project?rev=308069&view=rev
>>>> >> Log:
>>>> >> [libFuzzer] Allow non-fuzzer args after -ignore_remaining_args=1
>>>> >>
>>>> >> With this change, libFuzzer will ignore any arguments after a sigil
>>>> >> argument, but it will preserve these arguments at the end of the
>>>> >> command line when launching subprocesses. Using this, its possible to
>>>> >> handle positional and single-dash arguments to the program under test
>>>> >> by discarding everything up to -ignore_remaining_args=1 in
>>>> >> LLVMFuzzerInitialize.
>>>> >>
>>>> >> Added:
>>>> >> llvm/trunk/lib/Fuzzer/test/FlagsTest.cpp
>>>> >> Modified:
>>>> >> llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
>>>> >> llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>>>> >> llvm/trunk/lib/Fuzzer/FuzzerMerge.cpp
>>>> >> llvm/trunk/lib/Fuzzer/FuzzerUtil.h
>>>> >> llvm/trunk/lib/Fuzzer/test/CMakeLists.txt
>>>> >> llvm/trunk/lib/Fuzzer/test/fuzzer-flags.test
>>>> >>
>>>> >> Modified: llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
>>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>>> >> FuzzerDriver.cpp?rev=308069&r1=308068&r2=308069&view=diff
>>>> >> ============================================================
>>>> >> ==================
>>>> >> --- llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp (original)
>>>> >> +++ llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp Fri Jul 14 16:33:04 2017
>>>> >> @@ -186,7 +186,11 @@ static void ParseFlags(const std::vector
>>>> >> }
>>>> >> Inputs = new std::vector<std::string>;
>>>> >> for (size_t A = 1; A < Args.size(); A++) {
>>>> >> - if (ParseOneFlag(Args[A].c_str())) continue;
>>>> >> + if (ParseOneFlag(Args[A].c_str())) {
>>>> >> + if (Flags.ignore_remaining_args)
>>>> >> + break;
>>>> >> + continue;
>>>> >> + }
>>>> >> Inputs->push_back(Args[A]);
>>>> >> }
>>>> >> }
>>>> >> @@ -356,16 +360,17 @@ int MinimizeCrashInput(const std::vector
>>>> >> exit(1);
>>>> >> }
>>>> >> std::string InputFilePath = Inputs->at(0);
>>>> >> - std::string BaseCmd =
>>>> >> - CloneArgsWithoutX(Args, "minimize_crash",
>>>> "exact_artifact_path");
>>>> >> - auto InputPos = BaseCmd.find(" " + InputFilePath + " ");
>>>> >> + auto BaseCmd = SplitBefore(
>>>> >> + "-ignore_remaining_args=1",
>>>> >> + CloneArgsWithoutX(Args, "minimize_crash",
>>>> "exact_artifact_path"));
>>>> >> + auto InputPos = BaseCmd.first.find(" " + InputFilePath + " ");
>>>> >> assert(InputPos != std::string::npos);
>>>> >> - BaseCmd.erase(InputPos, InputFilePath.size() + 1);
>>>> >> + BaseCmd.first.erase(InputPos, InputFilePath.size() + 1);
>>>> >> if (Flags.runs <= 0 && Flags.max_total_time == 0) {
>>>> >> Printf("INFO: you need to specify -runs=N or "
>>>> >> "-max_total_time=N with -minimize_crash=1\n"
>>>> >> "INFO: defaulting to -max_total_time=600\n");
>>>> >> - BaseCmd += " -max_total_time=600";
>>>> >> + BaseCmd.first += " -max_total_time=600";
>>>> >> }
>>>> >>
>>>> >> auto LogFilePath = DirPlusFile(
>>>> >> @@ -378,7 +383,8 @@ int MinimizeCrashInput(const std::vector
>>>> >> Printf("CRASH_MIN: minimizing crash input: '%s' (%zd bytes)\n",
>>>> >> CurrentFilePath.c_str(), U.size());
>>>> >>
>>>> >> - auto Cmd = BaseCmd + " " + CurrentFilePath + LogFileRedirect;
>>>> >> + auto Cmd = BaseCmd.first + " " + CurrentFilePath +
>>>> LogFileRedirect +
>>>> >> " " +
>>>> >> + BaseCmd.second;
>>>> >>
>>>> >> Printf("CRASH_MIN: executing: %s\n", Cmd.c_str());
>>>> >> int ExitCode = ExecuteCommand(Cmd);
>>>> >>
>>>> >> Modified: llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>>> >> FuzzerFlags.def?rev=308069&r1=308068&r2=308069&view=diff
>>>> >> ============================================================
>>>> >> ==================
>>>> >> --- llvm/trunk/lib/Fuzzer/FuzzerFlags.def (original)
>>>> >> +++ llvm/trunk/lib/Fuzzer/FuzzerFlags.def Fri Jul 14 16:33:04 2017
>>>> >> @@ -121,6 +121,9 @@ FUZZER_FLAG_STRING(exit_on_src_pos, "Exi
>>>> >> FUZZER_FLAG_STRING(exit_on_item, "Exit if an item with a given
>>>> sha1 sum"
>>>> >> " was added to the corpus. "
>>>> >> "Used primarily for testing libFuzzer itself.")
>>>> >> +FUZZER_FLAG_INT(ignore_remaining_args, 0, "If 1, ignore all
>>>> arguments
>>>> >> passed "
>>>> >> + "after this one. Useful for fuzzers that need to do
>>>> their
>>>> >> own "
>>>> >> + "argument parsing.")
>>>> >>
>>>> >> FUZZER_FLAG_STRING(run_equivalence_server, "Experimental")
>>>> >> FUZZER_FLAG_STRING(use_equivalence_server, "Experimental")
>>>> >>
>>>> >> Modified: llvm/trunk/lib/Fuzzer/FuzzerMerge.cpp
>>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>>> >> FuzzerMerge.cpp?rev=308069&r1=308068&r2=308069&view=diff
>>>> >> ============================================================
>>>> >> ==================
>>>> >> --- llvm/trunk/lib/Fuzzer/FuzzerMerge.cpp (original)
>>>> >> +++ llvm/trunk/lib/Fuzzer/FuzzerMerge.cpp Fri Jul 14 16:33:04 2017
>>>> >> @@ -285,12 +285,13 @@ void Fuzzer::CrashResistantMerge(const s
>>>> >>
>>>> >> // Execute the inner process untill it passes.
>>>> >> // Every inner process should execute at least one input.
>>>> >> - std::string BaseCmd = CloneArgsWithoutX(Args, "keep-all-flags");
>>>> >> + auto BaseCmd = SplitBefore("-ignore_remaining_args=1",
>>>> >> + CloneArgsWithoutX(Args,
>>>> "keep-all-flags"));
>>>> >> bool Success = false;
>>>> >> for (size_t i = 1; i <= AllFiles.size(); i++) {
>>>> >> Printf("MERGE-OUTER: attempt %zd\n", i);
>>>> >> - auto ExitCode =
>>>> >> - ExecuteCommand(BaseCmd + " -merge_control_file=" + CFPath);
>>>> >> + auto ExitCode = ExecuteCommand(BaseCmd.first + "
>>>> >> -merge_control_file=" +
>>>> >> + CFPath + " " + BaseCmd.second);
>>>> >> if (!ExitCode) {
>>>> >> Printf("MERGE-OUTER: succesfull in %zd attempt(s)\n", i);
>>>> >> Success = true;
>>>> >>
>>>> >> Modified: llvm/trunk/lib/Fuzzer/FuzzerUtil.h
>>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>>> >> FuzzerUtil.h?rev=308069&r1=308068&r2=308069&view=diff
>>>> >> ============================================================
>>>> >> ==================
>>>> >> --- llvm/trunk/lib/Fuzzer/FuzzerUtil.h (original)
>>>> >> +++ llvm/trunk/lib/Fuzzer/FuzzerUtil.h Fri Jul 14 16:33:04 2017
>>>> >> @@ -67,6 +67,14 @@ inline std::string CloneArgsWithoutX(con
>>>> >> return CloneArgsWithoutX(Args, X, X);
>>>> >> }
>>>> >>
>>>> >> +inline std::pair<std::string, std::string> SplitBefore(std::string
>>>> X,
>>>> >> + std::string
>>>> S) {
>>>> >> + auto Pos = S.find(X);
>>>> >> + if (Pos == std::string::npos)
>>>> >> + return std::make_pair(S, "");
>>>> >> + return std::make_pair(S.substr(0, Pos), S.substr(Pos));
>>>> >> +}
>>>> >> +
>>>> >> std::string DisassembleCmd(const std::string &FileName);
>>>> >>
>>>> >> std::string SearchRegexCmd(const std::string &Regex);
>>>> >>
>>>> >> Modified: llvm/trunk/lib/Fuzzer/test/CMakeLists.txt
>>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>>> >> test/CMakeLists.txt?rev=308069&r1=308068&r2=308069&view=diff
>>>> >> ============================================================
>>>> >> ==================
>>>> >> --- llvm/trunk/lib/Fuzzer/test/CMakeLists.txt (original)
>>>> >> +++ llvm/trunk/lib/Fuzzer/test/CMakeLists.txt Fri Jul 14 16:33:04
>>>> 2017
>>>> >> @@ -90,6 +90,7 @@ set(Tests
>>>> >> EmptyTest
>>>> >> EquivalenceATest
>>>> >> EquivalenceBTest
>>>> >> + FlagsTest
>>>> >> FourIndependentBranchesTest
>>>> >> FullCoverageSetTest
>>>> >> InitializeTest
>>>> >>
>>>> >> Added: llvm/trunk/lib/Fuzzer/test/FlagsTest.cpp
>>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>>> >> test/FlagsTest.cpp?rev=308069&view=auto
>>>> >> ============================================================
>>>> >> ==================
>>>> >> --- llvm/trunk/lib/Fuzzer/test/FlagsTest.cpp (added)
>>>> >> +++ llvm/trunk/lib/Fuzzer/test/FlagsTest.cpp Fri Jul 14 16:33:04
>>>> 2017
>>>> >> @@ -0,0 +1,32 @@
>>>> >> +// This file is distributed under the University of Illinois Open
>>>> Source
>>>> >> +// License. See LICENSE.TXT for details.
>>>> >> +
>>>> >> +// Parse some flags
>>>> >> +#include <string>
>>>> >> +#include <vector>
>>>> >> +
>>>> >> +static std::vector<std::string> Flags;
>>>> >> +
>>>> >> +extern "C" int LLVMFuzzerInitialize(int *Argc, char ***Argv) {
>>>> >> + // Parse --flags and anything after -ignore_remaining_args=1 is
>>>> passed.
>>>> >> + int I = 1;
>>>> >> + while (I < *Argc) {
>>>> >> + std::string S((*Argv)[I++]);
>>>> >> + if (S == "-ignore_remaining_args=1")
>>>> >> + break;
>>>> >> + if (S.substr(0, 2) == "--")
>>>> >> + Flags.push_back(S);
>>>> >> + }
>>>> >> + while (I < *Argc)
>>>> >> + Flags.push_back(std::string((*Argv)[I++]));
>>>> >> +
>>>> >> + return 0;
>>>> >> +}
>>>> >> +
>>>> >> +extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t
>>>> Size) {
>>>> >> + fprintf(stderr, "BINGO ");
>>>> >> + for (auto Flag : Flags)
>>>> >> + fprintf(stderr, "%s ", Flag.c_str());
>>>> >> + fprintf(stderr, "\n");
>>>> >> + exit(0);
>>>> >> +}
>>>> >>
>>>> >> Modified: llvm/trunk/lib/Fuzzer/test/fuzzer-flags.test
>>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>>> >> test/fuzzer-flags.test?rev=308069&r1=308068&r2=308069&view=diff
>>>> >> ============================================================
>>>> >> ==================
>>>> >> --- llvm/trunk/lib/Fuzzer/test/fuzzer-flags.test (original)
>>>> >> +++ llvm/trunk/lib/Fuzzer/test/fuzzer-flags.test Fri Jul 14
>>>> 16:33:04 2017
>>>> >> @@ -1,10 +1,18 @@
>>>> >> -RUN: LLVMFuzzer-SimpleTest -foo_bar=1 2>&1 | FileCheck %s
>>>> >> --check-prefix=FOO_BAR
>>>> >> +RUN: LLVMFuzzer-FlagsTest -foo_bar=1 2>&1 | FileCheck %s
>>>> >> --check-prefix=FOO_BAR
>>>> >> FOO_BAR: WARNING: unrecognized flag '-foo_bar=1'; use -help=1 to
>>>> list all
>>>> >> flags
>>>> >> FOO_BAR: BINGO
>>>> >>
>>>> >> -RUN: LLVMFuzzer-SimpleTest -runs=10 --max_len=100 2>&1 | FileCheck
>>>> %s
>>>> >> --check-prefix=DASH_DASH
>>>> >> +RUN: LLVMFuzzer-FlagsTest -runs=10 --max_len=100 2>&1 | FileCheck %s
>>>> >> --check-prefix=DASH_DASH
>>>> >> DASH_DASH: WARNING: did you mean '-max_len=100' (single dash)?
>>>> >> DASH_DASH: INFO: A corpus is not provided, starting from an empty
>>>> corpus
>>>> >>
>>>> >> -RUN: LLVMFuzzer-SimpleTest -help=1 2>&1 | FileCheck %s
>>>> >> --check-prefix=NO_INTERNAL
>>>> >> +RUN: LLVMFuzzer-FlagsTest -help=1 2>&1 | FileCheck %s
>>>> >> --check-prefix=NO_INTERNAL
>>>> >> NO_INTERNAL-NOT: internal flag
>>>> >> +
>>>> >> +RUN: LLVMFuzzer-FlagsTest --foo-bar -runs=10
>>>> -ignore_remaining_args=1
>>>> >> --baz -help=1 test 2>&1 | FileCheck %s --check-prefix=PASSTHRU
>>>> >> +PASSTHRU: BINGO --foo-bar --baz -help=1 test
>>>> >> +
>>>> >> +RUN: mkdir -p %t/T0 %t/T1
>>>> >> +RUN: touch %t/T1/empty
>>>> >> +RUN: LLVMFuzzer-FlagsTest --foo-bar -merge=1 %t/T0 %t/T1
>>>> >> -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck %s
>>>> >> --check-prefix=PASSTHRU-MERGE
>>>> >> +PASSTHRU-MERGE: BINGO --foo-bar --baz -help=1 test
>>>> >>
>>>> >>
>>>> >> _______________________________________________
>>>> >> llvm-commits mailing list
>>>> >> llvm-commits at lists.llvm.org
>>>> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>> >>
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170719/00ac2b51/attachment.html>
More information about the llvm-commits
mailing list