<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 24, 2015 at 8:52 PM, Sean Silva <span dir="ltr"><<a href="mailto:chisophugis@gmail.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=chisophugis@gmail.com&cc=&bcc=&su=&body=','_blank');return false;">chisophugis@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Thu, Sep 24, 2015 at 7:57 PM, Kostya Serebryany <span dir="ltr"><<a href="mailto:kcc@google.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=kcc@google.com&cc=&bcc=&su=&body=','_blank');return false;">kcc@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Ah, sorry, this is google cloud storage. <div>Scales much better than git for this particular task. </div></div></blockquote><div><br></div></span><div>Makes sense. Out of curiosity how large is this bot's corpus?</div><span class=""><font color="#888888"><div><br></div></font></span></div></div></div></blockquote><div class="gmail_quote"><br></div><div class="gmail_quote">In bytes: </div><br>207M clang</div><div class="gmail_quote">84M clang-format</div><div class="gmail_quote">812M llvm-as<br><br></div>in # of inputs:</div><div class="gmail_extra"><br>clang 52015</div><div class="gmail_extra">clang-format 21113</div><div class="gmail_extra">llvm-as 201256<div class="gmail_quote"><br></div><div class="gmail_quote"><br></div><div class="gmail_quote">For clang and clang-format the corpora are reasonable,but not for llvm-as.</div><div class="gmail_quote">I've disabled llvm-as-fuzzer for now due to excessive parasitic coverage... </div><div class="gmail_quote"><a href="https://llvm.org/bugs/show_bug.cgi?id=24639#c10">https://llvm.org/bugs/show_bug.cgi?id=24639#c10</a><br></div><div class="gmail_quote"><br></div><div class="gmail_quote"> </div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><font color="#888888"><div></div><div>-- Sean Silva</div></font></span><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>--kcc </div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 24, 2015 at 7:02 PM, Sean Silva <span dir="ltr"><<a href="mailto:chisophugis@gmail.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=chisophugis@gmail.com&cc=&bcc=&su=&body=','_blank');return false;">chisophugis@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Thu, Sep 24, 2015 at 8:15 AM, Kostya Serebryany via llvm-commits <span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=llvm-commits@lists.llvm.org&cc=&bcc=&su=&body=','_blank');return false;">llvm-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Author: kcc<br>
Date: Thu Sep 24 10:15:15 2015<br>
New Revision: 248504<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=248504&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=248504&view=rev</a><br>
Log:<br>
[libFuzzer] take the fuzzing corpora from GCS instead of git<br></blockquote><div><br></div></span><div>What is GCS?</div><span><font color="#888888"><div><br></div><div>-- Sean Silva</div></font></span><div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Modified:<br>
    zorg/trunk/zorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh<br>
<br>
Modified: zorg/trunk/zorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh?rev=248504&r1=248503&r2=248504&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh?rev=248504&r1=248503&r2=248504&view=diff</a><br>
==============================================================================<br>
--- zorg/trunk/zorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh (original)<br>
+++ zorg/trunk/zorg/buildbot/builders/sanitizers/buildbot_fuzzer.sh Thu Sep 24 10:15:15 2015<br>
@@ -19,11 +19,22 @@ LLVM=$ROOT/llvm<br>
 # No assertions. Need to clean up the existing assertion failures first.<br>
 # Also, the Fuzzer does not provide reproducers on assertion failures yet.<br>
 CMAKE_COMMON_OPTIONS="-GNinja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_PARALLEL_LINK_JOBS=3"<br>
-CORPUS_ROOT=$ROOT/fuzzing-with-sanitizers/llvm<br>
+CORPUS_ROOT=$ROOT/CORPORA/llvm<br>
 CLANG_FORMAT_CORPUS=$CORPUS_ROOT/clang-format/C1<br>
 CLANG_CORPUS=$CORPUS_ROOT/clang/C1<br>
 LLVM_AS_CORPUS=$CORPUS_ROOT/llvm-as/C1<br>
<br>
+GS_ROOT=gs://fuzzing-with-sanitizers<br>
+<br>
+syncFromGs() {<br>
+  mkdir -p $CORPUS_ROOT/$1<br>
+  gsutil -m rsync $GS_ROOT/$1 $CORPUS_ROOT/$1<br>
+}<br>
+<br>
+syncToGs() {<br>
+  gsutil -m rsync $CORPUS_ROOT/$1 $GS_ROOT/$1<br>
+}<br>
+<br>
 if [ "$BUILDBOT_CLOBBER" != "" ]; then<br>
   echo @@@BUILD_STEP clobber@@@<br>
   rm -rf llvm<br>
@@ -36,12 +47,6 @@ fi<br>
 rm -rf ${STAGE2_ASAN_DIR}<br>
 rm -rf ${STAGE2_ASAN_ASSERTIONS_DIR}<br>
<br>
-# Create an empty directory for the corpus if it doesn't exist yet.<br>
-# It will get populated with examples.<br>
-# FIXME: synchronize this directory with some external persistent storage.<br>
-mkdir -p $CLANG_FORMAT_CORPUS<br>
-mkdir -p $CLANG_CORPUS<br>
-<br>
 # Make sure asan intercepts SIGABRT so that the fuzzer can print the test cases<br>
 # for assertion failures.<br>
 export ASAN_OPTIONS=handle_abort=1<br>
@@ -56,7 +61,9 @@ echo @@@BUILD_STEP build clang@@@<br>
 build_stage1_clang<br>
<br>
 echo @@@BUILD_STEP pull test corpuses @@@<br>
-(cd $CORPUS_ROOT && git pull --no-edit)<br>
+syncFromGs clang/C1<br>
+syncFromGs clang-format/C1<br>
+syncFromGs llvm-as/C1<br>
<br>
 # Stage 2 / AddressSanitizer<br>
<br>
@@ -126,5 +133,7 @@ echo @@@BUILD_STEP stage2/asan+assertion<br>
   echo @@@STEP_WARNINGS@@@<br>
<br>
 echo @@@BUILD_STEP push corpus updates@@@<br>
-$LLVM/lib/Fuzzer/pull_and_push_fuzz_corpus.sh $CORPUS_ROOT<br>
+syncToGs clang/C1<br>
+syncToGs clang-format/C1<br>
+syncToGs llvm-as/C1<br>
<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=llvm-commits@lists.llvm.org&cc=&bcc=&su=&body=','_blank');return false;">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div></div>