[llvm] r308069 - [libFuzzer] Allow non-fuzzer args after -ignore_remaining_args=1
Kostya Serebryany via llvm-commits
llvm-commits at lists.llvm.org
Mon Jul 17 17:58:48 PDT 2017
Let me add
UNSUPPORTED: windows
On Mon, Jul 17, 2017 at 5:50 PM, Reid Kleckner <rnk at google.com> wrote:
> I didn't have time to get to this today. I can look at it tomorrow. In the
> meantime, can somebody XFAIL this?
>
> On Mon, Jul 17, 2017 at 5:34 PM, Kostya Serebryany <kcc at google.com> wrote:
>
>> +rnk
>>
>> On Mon, Jul 17, 2017 at 12:25 PM, Justin Bogner <mail at justinbogner.com>
>> wrote:
>>
>>> The output on the bot isn't terribly informative and I don't have a
>>> windows machine. Could somebody run the failing command for me and paste
>>> the output? It's as follows:
>>>
>>> "LLVMFuzzer-FlagsTest" "--foo-bar" "-runs=10"
>>> "-ignore_remaining_args=1" "--baz" "-help=1" "test"
>>>
>>> Kostya Serebryany <kcc at google.com> writes:
>>> > This broke libFuzzer on Windows:
>>> > http://lab.llvm.org:8011/builders/sanitizer-windows/builds/1
>>> 4306/steps/run%20fuzzer%20tests/logs/stdio
>>> > I may not have time to look at this today, sadly.
>>> >
>>> > FAIL: LLVMFuzzer :: fuzzer-flags.test (14 of 92)
>>> > ******************** TEST 'LLVMFuzzer :: fuzzer-flags.test' FAILED
>>> > ********************
>>> > Script:
>>> > --
>>> > LLVMFuzzer-FlagsTest -foo_bar=1 2>&1 | FileCheck
>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>> > --check-prefix=FOO_BAR
>>> > LLVMFuzzer-FlagsTest -runs=10 --max_len=100 2>&1 | FileCheck
>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>> > --check-prefix=DASH_DASH
>>> > LLVMFuzzer-FlagsTest -help=1 2>&1 | FileCheck
>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>> > --check-prefix=NO_INTERNAL
>>> > LLVMFuzzer-FlagsTest --foo-bar -runs=10 -ignore_remaining_args=1 --baz
>>> > -help=1 test 2>&1 | FileCheck
>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>> > --check-prefix=PASSTHRU
>>> > mkdir -p
>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>> tput\fuzzer-flags.test.tmp/T0
>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>> tput\fuzzer-flags.test.tmp/T1
>>> > touch
>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>> tput\fuzzer-flags.test.tmp/T1/empty
>>> > LLVMFuzzer-FlagsTest --foo-bar -merge=1
>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>> tput\fuzzer-flags.test.tmp/T0
>>> > C:\b\slave\sanitizer-windows\build-fuzzer\lib\Fuzzer\test\Ou
>>> tput\fuzzer-flags.test.tmp/T1
>>> > -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck
>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test
>>> > --check-prefix=PASSTHRU-MERGE
>>> > --
>>> > Exit Code: 1
>>> >
>>> > Command Output (stdout):
>>> > --
>>> > $ "LLVMFuzzer-FlagsTest" "-foo_bar=1"
>>> > $ "FileCheck"
>>> > "C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test"
>>> > "--check-prefix=FOO_BAR"
>>> > $ "LLVMFuzzer-FlagsTest" "-runs=10" "--max_len=100"
>>> > $ "FileCheck"
>>> > "C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test"
>>> > "--check-prefix=DASH_DASH"
>>> > $ "LLVMFuzzer-FlagsTest" "-help=1"
>>> > $ "FileCheck"
>>> > "C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test"
>>> > "--check-prefix=NO_INTERNAL"
>>> > $ "LLVMFuzzer-FlagsTest" "--foo-bar" "-runs=10"
>>> > "-ignore_remaining_args=1" "--baz" "-help=1" "test"
>>> > $ "FileCheck"
>>> > "C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-flags.test"
>>> > "--check-prefix=PASSTHRU"
>>> > # command stderr:
>>> > C:\b\slave\sanitizer-windows\llvm\lib\Fuzzer\test\fuzzer-fla
>>> gs.test:13:11:
>>> > error: expected string not found in input
>>> >
>>> > PASSTHRU: BINGO --foo-bar --baz -help=1 test
>>> >
>>> > ^
>>> >
>>> > <stdin>:1:1: note: scanning from here
>>> >
>>> > INFO: libFuzzer ignores flags that start with '--'
>>> >
>>> > ^
>>> >
>>> >
>>> > error: command failed with exit status: 1
>>> >
>>> > --
>>> >
>>> > ********************
>>> > Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
>>> >
>>> >
>>> >
>>> > On Fri, Jul 14, 2017 at 4:33 PM, Justin Bogner via llvm-commits <
>>> > llvm-commits at lists.llvm.org> wrote:
>>> >
>>> >> Author: bogner
>>> >> Date: Fri Jul 14 16:33:04 2017
>>> >> New Revision: 308069
>>> >>
>>> >> URL: http://llvm.org/viewvc/llvm-project?rev=308069&view=rev
>>> >> Log:
>>> >> [libFuzzer] Allow non-fuzzer args after -ignore_remaining_args=1
>>> >>
>>> >> With this change, libFuzzer will ignore any arguments after a sigil
>>> >> argument, but it will preserve these arguments at the end of the
>>> >> command line when launching subprocesses. Using this, its possible to
>>> >> handle positional and single-dash arguments to the program under test
>>> >> by discarding everything up to -ignore_remaining_args=1 in
>>> >> LLVMFuzzerInitialize.
>>> >>
>>> >> Added:
>>> >> llvm/trunk/lib/Fuzzer/test/FlagsTest.cpp
>>> >> Modified:
>>> >> llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
>>> >> llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>>> >> llvm/trunk/lib/Fuzzer/FuzzerMerge.cpp
>>> >> llvm/trunk/lib/Fuzzer/FuzzerUtil.h
>>> >> llvm/trunk/lib/Fuzzer/test/CMakeLists.txt
>>> >> llvm/trunk/lib/Fuzzer/test/fuzzer-flags.test
>>> >>
>>> >> Modified: llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>> >> FuzzerDriver.cpp?rev=308069&r1=308068&r2=308069&view=diff
>>> >> ============================================================
>>> >> ==================
>>> >> --- llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp (original)
>>> >> +++ llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp Fri Jul 14 16:33:04 2017
>>> >> @@ -186,7 +186,11 @@ static void ParseFlags(const std::vector
>>> >> }
>>> >> Inputs = new std::vector<std::string>;
>>> >> for (size_t A = 1; A < Args.size(); A++) {
>>> >> - if (ParseOneFlag(Args[A].c_str())) continue;
>>> >> + if (ParseOneFlag(Args[A].c_str())) {
>>> >> + if (Flags.ignore_remaining_args)
>>> >> + break;
>>> >> + continue;
>>> >> + }
>>> >> Inputs->push_back(Args[A]);
>>> >> }
>>> >> }
>>> >> @@ -356,16 +360,17 @@ int MinimizeCrashInput(const std::vector
>>> >> exit(1);
>>> >> }
>>> >> std::string InputFilePath = Inputs->at(0);
>>> >> - std::string BaseCmd =
>>> >> - CloneArgsWithoutX(Args, "minimize_crash",
>>> "exact_artifact_path");
>>> >> - auto InputPos = BaseCmd.find(" " + InputFilePath + " ");
>>> >> + auto BaseCmd = SplitBefore(
>>> >> + "-ignore_remaining_args=1",
>>> >> + CloneArgsWithoutX(Args, "minimize_crash",
>>> "exact_artifact_path"));
>>> >> + auto InputPos = BaseCmd.first.find(" " + InputFilePath + " ");
>>> >> assert(InputPos != std::string::npos);
>>> >> - BaseCmd.erase(InputPos, InputFilePath.size() + 1);
>>> >> + BaseCmd.first.erase(InputPos, InputFilePath.size() + 1);
>>> >> if (Flags.runs <= 0 && Flags.max_total_time == 0) {
>>> >> Printf("INFO: you need to specify -runs=N or "
>>> >> "-max_total_time=N with -minimize_crash=1\n"
>>> >> "INFO: defaulting to -max_total_time=600\n");
>>> >> - BaseCmd += " -max_total_time=600";
>>> >> + BaseCmd.first += " -max_total_time=600";
>>> >> }
>>> >>
>>> >> auto LogFilePath = DirPlusFile(
>>> >> @@ -378,7 +383,8 @@ int MinimizeCrashInput(const std::vector
>>> >> Printf("CRASH_MIN: minimizing crash input: '%s' (%zd bytes)\n",
>>> >> CurrentFilePath.c_str(), U.size());
>>> >>
>>> >> - auto Cmd = BaseCmd + " " + CurrentFilePath + LogFileRedirect;
>>> >> + auto Cmd = BaseCmd.first + " " + CurrentFilePath +
>>> LogFileRedirect +
>>> >> " " +
>>> >> + BaseCmd.second;
>>> >>
>>> >> Printf("CRASH_MIN: executing: %s\n", Cmd.c_str());
>>> >> int ExitCode = ExecuteCommand(Cmd);
>>> >>
>>> >> Modified: llvm/trunk/lib/Fuzzer/FuzzerFlags.def
>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>> >> FuzzerFlags.def?rev=308069&r1=308068&r2=308069&view=diff
>>> >> ============================================================
>>> >> ==================
>>> >> --- llvm/trunk/lib/Fuzzer/FuzzerFlags.def (original)
>>> >> +++ llvm/trunk/lib/Fuzzer/FuzzerFlags.def Fri Jul 14 16:33:04 2017
>>> >> @@ -121,6 +121,9 @@ FUZZER_FLAG_STRING(exit_on_src_pos, "Exi
>>> >> FUZZER_FLAG_STRING(exit_on_item, "Exit if an item with a given sha1
>>> sum"
>>> >> " was added to the corpus. "
>>> >> "Used primarily for testing libFuzzer itself.")
>>> >> +FUZZER_FLAG_INT(ignore_remaining_args, 0, "If 1, ignore all
>>> arguments
>>> >> passed "
>>> >> + "after this one. Useful for fuzzers that need to do
>>> their
>>> >> own "
>>> >> + "argument parsing.")
>>> >>
>>> >> FUZZER_FLAG_STRING(run_equivalence_server, "Experimental")
>>> >> FUZZER_FLAG_STRING(use_equivalence_server, "Experimental")
>>> >>
>>> >> Modified: llvm/trunk/lib/Fuzzer/FuzzerMerge.cpp
>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>> >> FuzzerMerge.cpp?rev=308069&r1=308068&r2=308069&view=diff
>>> >> ============================================================
>>> >> ==================
>>> >> --- llvm/trunk/lib/Fuzzer/FuzzerMerge.cpp (original)
>>> >> +++ llvm/trunk/lib/Fuzzer/FuzzerMerge.cpp Fri Jul 14 16:33:04 2017
>>> >> @@ -285,12 +285,13 @@ void Fuzzer::CrashResistantMerge(const s
>>> >>
>>> >> // Execute the inner process untill it passes.
>>> >> // Every inner process should execute at least one input.
>>> >> - std::string BaseCmd = CloneArgsWithoutX(Args, "keep-all-flags");
>>> >> + auto BaseCmd = SplitBefore("-ignore_remaining_args=1",
>>> >> + CloneArgsWithoutX(Args,
>>> "keep-all-flags"));
>>> >> bool Success = false;
>>> >> for (size_t i = 1; i <= AllFiles.size(); i++) {
>>> >> Printf("MERGE-OUTER: attempt %zd\n", i);
>>> >> - auto ExitCode =
>>> >> - ExecuteCommand(BaseCmd + " -merge_control_file=" + CFPath);
>>> >> + auto ExitCode = ExecuteCommand(BaseCmd.first + "
>>> >> -merge_control_file=" +
>>> >> + CFPath + " " + BaseCmd.second);
>>> >> if (!ExitCode) {
>>> >> Printf("MERGE-OUTER: succesfull in %zd attempt(s)\n", i);
>>> >> Success = true;
>>> >>
>>> >> Modified: llvm/trunk/lib/Fuzzer/FuzzerUtil.h
>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>> >> FuzzerUtil.h?rev=308069&r1=308068&r2=308069&view=diff
>>> >> ============================================================
>>> >> ==================
>>> >> --- llvm/trunk/lib/Fuzzer/FuzzerUtil.h (original)
>>> >> +++ llvm/trunk/lib/Fuzzer/FuzzerUtil.h Fri Jul 14 16:33:04 2017
>>> >> @@ -67,6 +67,14 @@ inline std::string CloneArgsWithoutX(con
>>> >> return CloneArgsWithoutX(Args, X, X);
>>> >> }
>>> >>
>>> >> +inline std::pair<std::string, std::string> SplitBefore(std::string X,
>>> >> + std::string
>>> S) {
>>> >> + auto Pos = S.find(X);
>>> >> + if (Pos == std::string::npos)
>>> >> + return std::make_pair(S, "");
>>> >> + return std::make_pair(S.substr(0, Pos), S.substr(Pos));
>>> >> +}
>>> >> +
>>> >> std::string DisassembleCmd(const std::string &FileName);
>>> >>
>>> >> std::string SearchRegexCmd(const std::string &Regex);
>>> >>
>>> >> Modified: llvm/trunk/lib/Fuzzer/test/CMakeLists.txt
>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>> >> test/CMakeLists.txt?rev=308069&r1=308068&r2=308069&view=diff
>>> >> ============================================================
>>> >> ==================
>>> >> --- llvm/trunk/lib/Fuzzer/test/CMakeLists.txt (original)
>>> >> +++ llvm/trunk/lib/Fuzzer/test/CMakeLists.txt Fri Jul 14 16:33:04
>>> 2017
>>> >> @@ -90,6 +90,7 @@ set(Tests
>>> >> EmptyTest
>>> >> EquivalenceATest
>>> >> EquivalenceBTest
>>> >> + FlagsTest
>>> >> FourIndependentBranchesTest
>>> >> FullCoverageSetTest
>>> >> InitializeTest
>>> >>
>>> >> Added: llvm/trunk/lib/Fuzzer/test/FlagsTest.cpp
>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>> >> test/FlagsTest.cpp?rev=308069&view=auto
>>> >> ============================================================
>>> >> ==================
>>> >> --- llvm/trunk/lib/Fuzzer/test/FlagsTest.cpp (added)
>>> >> +++ llvm/trunk/lib/Fuzzer/test/FlagsTest.cpp Fri Jul 14 16:33:04 2017
>>> >> @@ -0,0 +1,32 @@
>>> >> +// This file is distributed under the University of Illinois Open
>>> Source
>>> >> +// License. See LICENSE.TXT for details.
>>> >> +
>>> >> +// Parse some flags
>>> >> +#include <string>
>>> >> +#include <vector>
>>> >> +
>>> >> +static std::vector<std::string> Flags;
>>> >> +
>>> >> +extern "C" int LLVMFuzzerInitialize(int *Argc, char ***Argv) {
>>> >> + // Parse --flags and anything after -ignore_remaining_args=1 is
>>> passed.
>>> >> + int I = 1;
>>> >> + while (I < *Argc) {
>>> >> + std::string S((*Argv)[I++]);
>>> >> + if (S == "-ignore_remaining_args=1")
>>> >> + break;
>>> >> + if (S.substr(0, 2) == "--")
>>> >> + Flags.push_back(S);
>>> >> + }
>>> >> + while (I < *Argc)
>>> >> + Flags.push_back(std::string((*Argv)[I++]));
>>> >> +
>>> >> + return 0;
>>> >> +}
>>> >> +
>>> >> +extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t
>>> Size) {
>>> >> + fprintf(stderr, "BINGO ");
>>> >> + for (auto Flag : Flags)
>>> >> + fprintf(stderr, "%s ", Flag.c_str());
>>> >> + fprintf(stderr, "\n");
>>> >> + exit(0);
>>> >> +}
>>> >>
>>> >> Modified: llvm/trunk/lib/Fuzzer/test/fuzzer-flags.test
>>> >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/
>>> >> test/fuzzer-flags.test?rev=308069&r1=308068&r2=308069&view=diff
>>> >> ============================================================
>>> >> ==================
>>> >> --- llvm/trunk/lib/Fuzzer/test/fuzzer-flags.test (original)
>>> >> +++ llvm/trunk/lib/Fuzzer/test/fuzzer-flags.test Fri Jul 14 16:33:04
>>> 2017
>>> >> @@ -1,10 +1,18 @@
>>> >> -RUN: LLVMFuzzer-SimpleTest -foo_bar=1 2>&1 | FileCheck %s
>>> >> --check-prefix=FOO_BAR
>>> >> +RUN: LLVMFuzzer-FlagsTest -foo_bar=1 2>&1 | FileCheck %s
>>> >> --check-prefix=FOO_BAR
>>> >> FOO_BAR: WARNING: unrecognized flag '-foo_bar=1'; use -help=1 to
>>> list all
>>> >> flags
>>> >> FOO_BAR: BINGO
>>> >>
>>> >> -RUN: LLVMFuzzer-SimpleTest -runs=10 --max_len=100 2>&1 | FileCheck %s
>>> >> --check-prefix=DASH_DASH
>>> >> +RUN: LLVMFuzzer-FlagsTest -runs=10 --max_len=100 2>&1 | FileCheck %s
>>> >> --check-prefix=DASH_DASH
>>> >> DASH_DASH: WARNING: did you mean '-max_len=100' (single dash)?
>>> >> DASH_DASH: INFO: A corpus is not provided, starting from an empty
>>> corpus
>>> >>
>>> >> -RUN: LLVMFuzzer-SimpleTest -help=1 2>&1 | FileCheck %s
>>> >> --check-prefix=NO_INTERNAL
>>> >> +RUN: LLVMFuzzer-FlagsTest -help=1 2>&1 | FileCheck %s
>>> >> --check-prefix=NO_INTERNAL
>>> >> NO_INTERNAL-NOT: internal flag
>>> >> +
>>> >> +RUN: LLVMFuzzer-FlagsTest --foo-bar -runs=10 -ignore_remaining_args=1
>>> >> --baz -help=1 test 2>&1 | FileCheck %s --check-prefix=PASSTHRU
>>> >> +PASSTHRU: BINGO --foo-bar --baz -help=1 test
>>> >> +
>>> >> +RUN: mkdir -p %t/T0 %t/T1
>>> >> +RUN: touch %t/T1/empty
>>> >> +RUN: LLVMFuzzer-FlagsTest --foo-bar -merge=1 %t/T0 %t/T1
>>> >> -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck %s
>>> >> --check-prefix=PASSTHRU-MERGE
>>> >> +PASSTHRU-MERGE: BINGO --foo-bar --baz -help=1 test
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> llvm-commits mailing list
>>> >> llvm-commits at lists.llvm.org
>>> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>> >>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170717/9a332f64/attachment-0001.html>
More information about the llvm-commits
mailing list