<div dir="ltr">+rnk</div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 17, 2017 at 12:25 PM, Justin Bogner <span dir="ltr"><<a href="mailto:mail@justinbogner.com" target="_blank">mail@justinbogner.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The output on the bot isn't terribly informative and I don't have a<br>
windows machine. Could somebody run the failing command for me and paste<br>
the output? It's as follows:<br>
<span class="im HOEnZb"><br>
"LLVMFuzzer-FlagsTest" "--foo-bar" "-runs=10" "-ignore_remaining_args=1" "--baz" "-help=1" "test"<br>
<br>
</span><div class="HOEnZb"><div class="h5">Kostya Serebryany <<a href="mailto:kcc@google.com">kcc@google.com</a>> writes:<br>
> This broke libFuzzer on Windows:<br>
> <a href="http://lab.llvm.org:8011/builders/sanitizer-windows/builds/14306/steps/run%20fuzzer%20tests/logs/stdio" rel="noreferrer" target="_blank">http://lab.llvm.org:8011/<wbr>builders/sanitizer-windows/<wbr>builds/14306/steps/run%<wbr>20fuzzer%20tests/logs/stdio</a><br>
> I may not have time to look at this today, sadly.<br>
><br>
> FAIL: LLVMFuzzer :: fuzzer-flags.test (14 of 92)<br>
> ******************** TEST 'LLVMFuzzer :: fuzzer-flags.test' FAILED<br>
> ********************<br>
> Script:<br>
> --<br>
> LLVMFuzzer-FlagsTest -foo_bar=1 2>&1 | FileCheck<br>
> C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test<br>
> --check-prefix=FOO_BAR<br>
> LLVMFuzzer-FlagsTest -runs=10 --max_len=100 2>&1 | FileCheck<br>
> C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test<br>
> --check-prefix=DASH_DASH<br>
> LLVMFuzzer-FlagsTest -help=1 2>&1 | FileCheck<br>
> C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test<br>
> --check-prefix=NO_INTERNAL<br>
> LLVMFuzzer-FlagsTest --foo-bar -runs=10 -ignore_remaining_args=1 --baz<br>
> -help=1 test 2>&1 | FileCheck<br>
> C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test<br>
> --check-prefix=PASSTHRU<br>
> mkdir -p<br>
> C:\b\slave\sanitizer-windows\<wbr>build-fuzzer\lib\Fuzzer\test\<wbr>Output\fuzzer-flags.test.tmp/<wbr>T0<br>
> C:\b\slave\sanitizer-windows\<wbr>build-fuzzer\lib\Fuzzer\test\<wbr>Output\fuzzer-flags.test.tmp/<wbr>T1<br>
> touch<br>
> C:\b\slave\sanitizer-windows\<wbr>build-fuzzer\lib\Fuzzer\test\<wbr>Output\fuzzer-flags.test.tmp/<wbr>T1/empty<br>
> LLVMFuzzer-FlagsTest --foo-bar -merge=1<br>
> C:\b\slave\sanitizer-windows\<wbr>build-fuzzer\lib\Fuzzer\test\<wbr>Output\fuzzer-flags.test.tmp/<wbr>T0<br>
> C:\b\slave\sanitizer-windows\<wbr>build-fuzzer\lib\Fuzzer\test\<wbr>Output\fuzzer-flags.test.tmp/<wbr>T1<br>
> -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck<br>
> C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test<br>
> --check-prefix=PASSTHRU-MERGE<br>
> --<br>
> Exit Code: 1<br>
><br>
> Command Output (stdout):<br>
> --<br>
> $ "LLVMFuzzer-FlagsTest" "-foo_bar=1"<br>
> $ "FileCheck"<br>
> "C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test"<br>
> "--check-prefix=FOO_BAR"<br>
> $ "LLVMFuzzer-FlagsTest" "-runs=10" "--max_len=100"<br>
> $ "FileCheck"<br>
> "C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test"<br>
> "--check-prefix=DASH_DASH"<br>
> $ "LLVMFuzzer-FlagsTest" "-help=1"<br>
> $ "FileCheck"<br>
> "C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test"<br>
> "--check-prefix=NO_INTERNAL"<br>
> $ "LLVMFuzzer-FlagsTest" "--foo-bar" "-runs=10"<br>
> "-ignore_remaining_args=1" "--baz" "-help=1" "test"<br>
> $ "FileCheck"<br>
> "C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test"<br>
> "--check-prefix=PASSTHRU"<br>
> # command stderr:<br>
> C:\b\slave\sanitizer-windows\<wbr>llvm\lib\Fuzzer\test\fuzzer-<wbr>flags.test:13:11:<br>
> error: expected string not found in input<br>
><br>
> PASSTHRU: BINGO --foo-bar --baz -help=1 test<br>
><br>
> ^<br>
><br>
> <stdin>:1:1: note: scanning from here<br>
><br>
> INFO: libFuzzer ignores flags that start with '--'<br>
><br>
> ^<br>
><br>
><br>
> error: command failed with exit status: 1<br>
><br>
> --<br>
><br>
> ********************<br>
> Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..<br>
><br>
><br>
><br>
> On Fri, Jul 14, 2017 at 4:33 PM, Justin Bogner via llvm-commits <<br>
> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> wrote:<br>
><br>
>> Author: bogner<br>
>> Date: Fri Jul 14 16:33:04 2017<br>
>> New Revision: 308069<br>
>><br>
>> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=308069&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project?rev=308069&view=rev</a><br>
>> Log:<br>
>> [libFuzzer] Allow non-fuzzer args after -ignore_remaining_args=1<br>
>><br>
>> With this change, libFuzzer will ignore any arguments after a sigil<br>
>> argument, but it will preserve these arguments at the end of the<br>
>> command line when launching subprocesses. Using this, its possible to<br>
>> handle positional and single-dash arguments to the program under test<br>
>> by discarding everything up to -ignore_remaining_args=1 in<br>
>> LLVMFuzzerInitialize.<br>
>><br>
>> Added:<br>
>> llvm/trunk/lib/Fuzzer/test/<wbr>FlagsTest.cpp<br>
>> Modified:<br>
>> llvm/trunk/lib/Fuzzer/<wbr>FuzzerDriver.cpp<br>
>> llvm/trunk/lib/Fuzzer/<wbr>FuzzerFlags.def<br>
>> llvm/trunk/lib/Fuzzer/<wbr>FuzzerMerge.cpp<br>
>> llvm/trunk/lib/Fuzzer/<wbr>FuzzerUtil.h<br>
>> llvm/trunk/lib/Fuzzer/test/<wbr>CMakeLists.txt<br>
>> llvm/trunk/lib/Fuzzer/test/<wbr>fuzzer-flags.test<br>
>><br>
>> Modified: llvm/trunk/lib/Fuzzer/<wbr>FuzzerDriver.cpp<br>
>> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/Fuzzer/</a><br>
>> FuzzerDriver.cpp?rev=308069&<wbr>r1=308068&r2=308069&view=diff<br>
>> ==============================<wbr>==============================<br>
>> ==================<br>
>> --- llvm/trunk/lib/Fuzzer/<wbr>FuzzerDriver.cpp (original)<br>
>> +++ llvm/trunk/lib/Fuzzer/<wbr>FuzzerDriver.cpp Fri Jul 14 16:33:04 2017<br>
>> @@ -186,7 +186,11 @@ static void ParseFlags(const std::vector<br>
>> }<br>
>> Inputs = new std::vector<std::string>;<br>
>> for (size_t A = 1; A < Args.size(); A++) {<br>
>> - if (ParseOneFlag(Args[A].c_str())<wbr>) continue;<br>
>> + if (ParseOneFlag(Args[A].c_str())<wbr>) {<br>
>> + if (Flags.ignore_remaining_args)<br>
>> + break;<br>
>> + continue;<br>
>> + }<br>
>> Inputs->push_back(Args[A]);<br>
>> }<br>
>> }<br>
>> @@ -356,16 +360,17 @@ int MinimizeCrashInput(const std::vector<br>
>> exit(1);<br>
>> }<br>
>> std::string InputFilePath = Inputs->at(0);<br>
>> - std::string BaseCmd =<br>
>> - CloneArgsWithoutX(Args, "minimize_crash", "exact_artifact_path");<br>
>> - auto InputPos = BaseCmd.find(" " + InputFilePath + " ");<br>
>> + auto BaseCmd = SplitBefore(<br>
>> + "-ignore_remaining_args=1",<br>
>> + CloneArgsWithoutX(Args, "minimize_crash", "exact_artifact_path"));<br>
>> + auto InputPos = BaseCmd.first.find(" " + InputFilePath + " ");<br>
>> assert(InputPos != std::string::npos);<br>
>> - BaseCmd.erase(InputPos, InputFilePath.size() + 1);<br>
>> + BaseCmd.first.erase(InputPos, InputFilePath.size() + 1);<br>
>> if (Flags.runs <= 0 && Flags.max_total_time == 0) {<br>
>> Printf("INFO: you need to specify -runs=N or "<br>
>> "-max_total_time=N with -minimize_crash=1\n"<br>
>> "INFO: defaulting to -max_total_time=600\n");<br>
>> - BaseCmd += " -max_total_time=600";<br>
>> + BaseCmd.first += " -max_total_time=600";<br>
>> }<br>
>><br>
>> auto LogFilePath = DirPlusFile(<br>
>> @@ -378,7 +383,8 @@ int MinimizeCrashInput(const std::vector<br>
>> Printf("CRASH_MIN: minimizing crash input: '%s' (%zd bytes)\n",<br>
>> CurrentFilePath.c_str(), U.size());<br>
>><br>
>> - auto Cmd = BaseCmd + " " + CurrentFilePath + LogFileRedirect;<br>
>> + auto Cmd = BaseCmd.first + " " + CurrentFilePath + LogFileRedirect +<br>
>> " " +<br>
>> + BaseCmd.second;<br>
>><br>
>> Printf("CRASH_MIN: executing: %s\n", Cmd.c_str());<br>
>> int ExitCode = ExecuteCommand(Cmd);<br>
>><br>
>> Modified: llvm/trunk/lib/Fuzzer/<wbr>FuzzerFlags.def<br>
>> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/Fuzzer/</a><br>
>> FuzzerFlags.def?rev=308069&r1=<wbr>308068&r2=308069&view=diff<br>
>> ==============================<wbr>==============================<br>
>> ==================<br>
>> --- llvm/trunk/lib/Fuzzer/<wbr>FuzzerFlags.def (original)<br>
>> +++ llvm/trunk/lib/Fuzzer/<wbr>FuzzerFlags.def Fri Jul 14 16:33:04 2017<br>
>> @@ -121,6 +121,9 @@ FUZZER_FLAG_STRING(exit_on_<wbr>src_pos, "Exi<br>
>> FUZZER_FLAG_STRING(exit_on_<wbr>item, "Exit if an item with a given sha1 sum"<br>
>> " was added to the corpus. "<br>
>> "Used primarily for testing libFuzzer itself.")<br>
>> +FUZZER_FLAG_INT(ignore_<wbr>remaining_args, 0, "If 1, ignore all arguments<br>
>> passed "<br>
>> + "after this one. Useful for fuzzers that need to do their<br>
>> own "<br>
>> + "argument parsing.")<br>
>><br>
>> FUZZER_FLAG_STRING(run_<wbr>equivalence_server, "Experimental")<br>
>> FUZZER_FLAG_STRING(use_<wbr>equivalence_server, "Experimental")<br>
>><br>
>> Modified: llvm/trunk/lib/Fuzzer/<wbr>FuzzerMerge.cpp<br>
>> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/Fuzzer/</a><br>
>> FuzzerMerge.cpp?rev=308069&r1=<wbr>308068&r2=308069&view=diff<br>
>> ==============================<wbr>==============================<br>
>> ==================<br>
>> --- llvm/trunk/lib/Fuzzer/<wbr>FuzzerMerge.cpp (original)<br>
>> +++ llvm/trunk/lib/Fuzzer/<wbr>FuzzerMerge.cpp Fri Jul 14 16:33:04 2017<br>
>> @@ -285,12 +285,13 @@ void Fuzzer::CrashResistantMerge(<wbr>const s<br>
>><br>
>> // Execute the inner process untill it passes.<br>
>> // Every inner process should execute at least one input.<br>
>> - std::string BaseCmd = CloneArgsWithoutX(Args, "keep-all-flags");<br>
>> + auto BaseCmd = SplitBefore("-ignore_<wbr>remaining_args=1",<br>
>> + CloneArgsWithoutX(Args, "keep-all-flags"));<br>
>> bool Success = false;<br>
>> for (size_t i = 1; i <= AllFiles.size(); i++) {<br>
>> Printf("MERGE-OUTER: attempt %zd\n", i);<br>
>> - auto ExitCode =<br>
>> - ExecuteCommand(BaseCmd + " -merge_control_file=" + CFPath);<br>
>> + auto ExitCode = ExecuteCommand(BaseCmd.first + "<br>
>> -merge_control_file=" +<br>
>> + CFPath + " " + BaseCmd.second);<br>
>> if (!ExitCode) {<br>
>> Printf("MERGE-OUTER: succesfull in %zd attempt(s)\n", i);<br>
>> Success = true;<br>
>><br>
>> Modified: llvm/trunk/lib/Fuzzer/<wbr>FuzzerUtil.h<br>
>> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/Fuzzer/</a><br>
>> FuzzerUtil.h?rev=308069&r1=<wbr>308068&r2=308069&view=diff<br>
>> ==============================<wbr>==============================<br>
>> ==================<br>
>> --- llvm/trunk/lib/Fuzzer/<wbr>FuzzerUtil.h (original)<br>
>> +++ llvm/trunk/lib/Fuzzer/<wbr>FuzzerUtil.h Fri Jul 14 16:33:04 2017<br>
>> @@ -67,6 +67,14 @@ inline std::string CloneArgsWithoutX(con<br>
>> return CloneArgsWithoutX(Args, X, X);<br>
>> }<br>
>><br>
>> +inline std::pair<std::string, std::string> SplitBefore(std::string X,<br>
>> + std::string S) {<br>
>> + auto Pos = S.find(X);<br>
>> + if (Pos == std::string::npos)<br>
>> + return std::make_pair(S, "");<br>
>> + return std::make_pair(S.substr(0, Pos), S.substr(Pos));<br>
>> +}<br>
>> +<br>
>> std::string DisassembleCmd(const std::string &FileName);<br>
>><br>
>> std::string SearchRegexCmd(const std::string &Regex);<br>
>><br>
>> Modified: llvm/trunk/lib/Fuzzer/test/<wbr>CMakeLists.txt<br>
>> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/Fuzzer/</a><br>
>> test/CMakeLists.txt?rev=<wbr>308069&r1=308068&r2=308069&<wbr>view=diff<br>
>> ==============================<wbr>==============================<br>
>> ==================<br>
>> --- llvm/trunk/lib/Fuzzer/test/<wbr>CMakeLists.txt (original)<br>
>> +++ llvm/trunk/lib/Fuzzer/test/<wbr>CMakeLists.txt Fri Jul 14 16:33:04 2017<br>
>> @@ -90,6 +90,7 @@ set(Tests<br>
>> EmptyTest<br>
>> EquivalenceATest<br>
>> EquivalenceBTest<br>
>> + FlagsTest<br>
>> FourIndependentBranchesTest<br>
>> FullCoverageSetTest<br>
>> InitializeTest<br>
>><br>
>> Added: llvm/trunk/lib/Fuzzer/test/<wbr>FlagsTest.cpp<br>
>> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/Fuzzer/</a><br>
>> test/FlagsTest.cpp?rev=308069&<wbr>view=auto<br>
>> ==============================<wbr>==============================<br>
>> ==================<br>
>> --- llvm/trunk/lib/Fuzzer/test/<wbr>FlagsTest.cpp (added)<br>
>> +++ llvm/trunk/lib/Fuzzer/test/<wbr>FlagsTest.cpp Fri Jul 14 16:33:04 2017<br>
>> @@ -0,0 +1,32 @@<br>
>> +// This file is distributed under the University of Illinois Open Source<br>
>> +// License. See LICENSE.TXT for details.<br>
>> +<br>
>> +// Parse some flags<br>
>> +#include <string><br>
>> +#include <vector><br>
>> +<br>
>> +static std::vector<std::string> Flags;<br>
>> +<br>
>> +extern "C" int LLVMFuzzerInitialize(int *Argc, char ***Argv) {<br>
>> + // Parse --flags and anything after -ignore_remaining_args=1 is passed.<br>
>> + int I = 1;<br>
>> + while (I < *Argc) {<br>
>> + std::string S((*Argv)[I++]);<br>
>> + if (S == "-ignore_remaining_args=1")<br>
>> + break;<br>
>> + if (S.substr(0, 2) == "--")<br>
>> + Flags.push_back(S);<br>
>> + }<br>
>> + while (I < *Argc)<br>
>> + Flags.push_back(std::string((*<wbr>Argv)[I++]));<br>
>> +<br>
>> + return 0;<br>
>> +}<br>
>> +<br>
>> +extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {<br>
>> + fprintf(stderr, "BINGO ");<br>
>> + for (auto Flag : Flags)<br>
>> + fprintf(stderr, "%s ", Flag.c_str());<br>
>> + fprintf(stderr, "\n");<br>
>> + exit(0);<br>
>> +}<br>
>><br>
>> Modified: llvm/trunk/lib/Fuzzer/test/<wbr>fuzzer-flags.test<br>
>> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/Fuzzer/</a><br>
>> test/fuzzer-flags.test?rev=<wbr>308069&r1=308068&r2=308069&<wbr>view=diff<br>
>> ==============================<wbr>==============================<br>
>> ==================<br>
>> --- llvm/trunk/lib/Fuzzer/test/<wbr>fuzzer-flags.test (original)<br>
>> +++ llvm/trunk/lib/Fuzzer/test/<wbr>fuzzer-flags.test Fri Jul 14 16:33:04 2017<br>
>> @@ -1,10 +1,18 @@<br>
>> -RUN: LLVMFuzzer-SimpleTest -foo_bar=1 2>&1 | FileCheck %s<br>
>> --check-prefix=FOO_BAR<br>
>> +RUN: LLVMFuzzer-FlagsTest -foo_bar=1 2>&1 | FileCheck %s<br>
>> --check-prefix=FOO_BAR<br>
>> FOO_BAR: WARNING: unrecognized flag '-foo_bar=1'; use -help=1 to list all<br>
>> flags<br>
>> FOO_BAR: BINGO<br>
>><br>
>> -RUN: LLVMFuzzer-SimpleTest -runs=10 --max_len=100 2>&1 | FileCheck %s<br>
>> --check-prefix=DASH_DASH<br>
>> +RUN: LLVMFuzzer-FlagsTest -runs=10 --max_len=100 2>&1 | FileCheck %s<br>
>> --check-prefix=DASH_DASH<br>
>> DASH_DASH: WARNING: did you mean '-max_len=100' (single dash)?<br>
>> DASH_DASH: INFO: A corpus is not provided, starting from an empty corpus<br>
>><br>
>> -RUN: LLVMFuzzer-SimpleTest -help=1 2>&1 | FileCheck %s<br>
>> --check-prefix=NO_INTERNAL<br>
>> +RUN: LLVMFuzzer-FlagsTest -help=1 2>&1 | FileCheck %s<br>
>> --check-prefix=NO_INTERNAL<br>
>> NO_INTERNAL-NOT: internal flag<br>
>> +<br>
>> +RUN: LLVMFuzzer-FlagsTest --foo-bar -runs=10 -ignore_remaining_args=1<br>
>> --baz -help=1 test 2>&1 | FileCheck %s --check-prefix=PASSTHRU<br>
>> +PASSTHRU: BINGO --foo-bar --baz -help=1 test<br>
>> +<br>
>> +RUN: mkdir -p %t/T0 %t/T1<br>
>> +RUN: touch %t/T1/empty<br>
>> +RUN: LLVMFuzzer-FlagsTest --foo-bar -merge=1 %t/T0 %t/T1<br>
>> -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck %s<br>
>> --check-prefix=PASSTHRU-MERGE<br>
>> +PASSTHRU-MERGE: BINGO --foo-bar --baz -help=1 test<br>
>><br>
>><br>
>> ______________________________<wbr>_________________<br>
>> llvm-commits mailing list<br>
>> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-commits</a><br>
>><br>
</div></div></blockquote></div><br></div>