<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:DengXian;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
/* Page Definitions */
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="ZH-CN" link="blue" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi,</span></p>
<p class="MsoNormal"><span lang="EN-US">I’ve created an initial version of my GSoC 2021 Proposal and would like to hear your comments! Since it is an early draft, I would like to improve it over the following days. If you’re concerned about the scope of this
proposal being too large or too small or there details that needs further explanation, feel free to add comments or mail me!</span></p>
<p class="MsoNormal"><span lang="EN-US">I mentioned implementing mutations as LLVM IR passes in the proposal. There are some benefits that I can think of, not really sure if it will work out in the end... I’ve also posted a question on the LibFuzzer google
group regarding how mutations are scheduled when a custom mutator is used, but haven’t received a reply yet. Would like to hear your opinion and advice!</span></p>
<p class="MsoNormal"><span lang="EN-US">Link is here (google doc): <a href="https://docs.google.com/document/d/1kKr5IFQVhGCPspVESv4muhqZhdnU7YLPohRn50g5sgY/edit?usp=sharing">
https://docs.google.com/document/d/1kKr5IFQVhGCPspVESv4muhqZhdnU7YLPohRn50g5sgY/edit?usp=sharing</a></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Best,</span></p>
<p class="MsoNormal"><span lang="EN-US">Chibin Zhang</span></p>
<p class="MsoNormal"><span lang="EN-US">2021.3.31</span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:SimSun"><o:p> </o:p></span></p>
<div style="mso-element:para-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="border:none;padding:0cm"><b>发件人<span lang="EN-US">: </span>
</b><span lang="EN-US"><a href="mailto:johannesdoerfert@gmail.com">Johannes Doerfert</a><br>
</span><b>发送时间<span lang="EN-US">: </span></b><span lang="EN-US">2021</span>年<span lang="EN-US">3</span>月<span lang="EN-US">11</span>日<span lang="EN-US"> 0:53<br>
</span><b>收件人<span lang="EN-US">: </span></b><span lang="EN-US"><a href="mailto:zhangchb1@shanghaitech.edu.cn"><span lang="EN-US"><span lang="EN-US">张驰斌</span></span></a>;
<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a><br>
</span><b>主题<span lang="EN-US">: </span></b><span lang="EN-US">Re: [llvm-dev] Applying for GSoC 2021(Fuzzing LLVM-IR Passes)</span></p>
</div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:SimSun"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Hi Chibin,</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">On 3/9/21 3:12 AM, </span>张驰斌<span lang="EN-US"> wrote:</span></p>
<p class="MsoNormal"><span lang="EN-US">> Hi Johannes,</span></p>
<p class="MsoNormal"><span lang="EN-US">> Glad to hear from you! I understand that the title listed in the llvm GSoC 2021 webpage serves as a general guideline but a project proposal might need limit its scope and focus on the deliverables.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Yes, students will write the actual proposal which should contain more
</span></p>
<p class="MsoNormal"><span lang="EN-US">details and scope discussion.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> The ideas proposed all seems quite appealing and relevant to me. I’ve been browsing through llvm.rog/docs/FuzzingLLVM.html and llvm-project/*/tools/*-fuzzer recently as well as the youtube video that you mentioned
on the GSoC site. The following are some questions I’ve accumulated. (forgive me if they are too naïve…).</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Questions are always good.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> 1. Truth to be told, I’ve used OpenMP before for my course project, but I haven’t look into the inner workings of it, e.g. how it actually instruments programs decorated with #pragma, and how it interact with the
OS’s threading. If llvm’s OpenMP implementation hasn’t been fuzzed before, then it surely is a valuable fuzz target. Could you give some clue on how we could fuzz OpenMP? Like writing a parser for fuzzer input and calling openmp library function in LLVMFuzzOneInput
function? Or we fuzz it through clang? I’ll look into llvm-project/openmp some more.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">So the OpenMP runtime has an "internal" and an external part. The
</span></p>
<p class="MsoNormal"><span lang="EN-US">internal part is full of undocumented dependences so I doubt we can fuzz
</span></p>
<p class="MsoNormal"><span lang="EN-US">it without breaking at least one for each test. The external one is
</span></p>
<p class="MsoNormal"><span lang="EN-US">fuzzable however. That said, generating OpenMP programs to be feed to
</span></p>
<p class="MsoNormal"><span lang="EN-US">clang seems like a good thing to do. OpenMP has it's own set of
</span></p>
<p class="MsoNormal"><span lang="EN-US">"documented" dependences, e.g., nesting restrictions, but that is not
</span></p>
<p class="MsoNormal"><span lang="EN-US">necessarily a problem.</span></p>
<p class="MsoNormal"><span lang="EN-US">If we generate an invalid OpenMP program we should gracefully fail, in
</span></p>
<p class="MsoNormal"><span lang="EN-US">most cases. If we don't we have good test cases for an OpenMP sanitizer
</span></p>
<p class="MsoNormal"><span lang="EN-US">later on. We could also embed knowledge about nesting and other OpenMP
</span></p>
<p class="MsoNormal"><span lang="EN-US">restrictions into the fuzzer/mutation tester/test generator. Long story
</span></p>
<p class="MsoNormal"><span lang="EN-US">short, generating a large corpus of OpenMP inputs is certainly something
</span></p>
<p class="MsoNormal"><span lang="EN-US">I'm interested in, we can start with "random" programs and evolve
</span></p>
<p class="MsoNormal"><span lang="EN-US">towards more targeted approaches.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> 2. For the custom mutator idea. My understanding is that currently there are 2 kinds of mutators, the generic one that is shipped with LibFuzzer (Bit flipping, splicing, etc.), and a structural mutator. Is the
structural mutator related to IRMutator.cpp in the FuzzMutate folder?</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I'm not sure myself. I think "structural" here means it fuzzes a well
</span></p>
<p class="MsoNormal"><span lang="EN-US">defined structures, here protobuf. I might be wrong.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">What I was looking for, among other things, is a way to do CFG
</span></p>
<p class="MsoNormal"><span lang="EN-US">transformations and less obvious IR transformations, maybe:</span></p>
<p class="MsoNormal"><span lang="EN-US"> - Add a "while-loop" with one iteration around a (set of) block(s)
</span></p>
<p class="MsoNormal"><span lang="EN-US">(various ways to "hide" the one iteration part)</span></p>
<p class="MsoNormal"><span lang="EN-US"> - Add a "do-loop" with zero iterations around a (set of) block(s)
</span></p>
<p class="MsoNormal"><span lang="EN-US">(various ways to "hide" the zero iterations part)</span></p>
<p class="MsoNormal"><span lang="EN-US"> - Add a call to an function SCC which does effectively nothing but
</span></p>
<p class="MsoNormal"><span lang="EN-US">writes new buffers passed to it or allocated within.</span></p>
<p class="MsoNormal"><span lang="EN-US"> - Add branches that will not be taken with various targets,
</span></p>
<p class="MsoNormal"><span lang="EN-US">unreachable, some arbitrary block in the function, etc.</span></p>
<p class="MsoNormal"><span lang="EN-US"> - Add arguments to functions that are effectively useless.</span></p>
<p class="MsoNormal"><span lang="EN-US"> - ...</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We would do those and record if and how the change impacted passes or
</span></p>
<p class="MsoNormal"><span lang="EN-US">the entire O3 pipeline. Learn about our heuristics and cutoffs and such,
</span></p>
<p class="MsoNormal"><span lang="EN-US">build a database, etc.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> 3. Most of the bugs found by fuzzers are usually crashes or hangs. Correctness testing is interesting but hard to achieve from my limited knowledge. I wonder if this is related to the ‘Alive’ tool mentioned by
Florian? The fuzzer provides input to some llvm pass, and ‘Alive’ will verify that the transformation is valid. Please correct me if my understanding is wrong…</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Yes, that is the idea. If we fuzz blindly, as opposed to guided test
</span></p>
<p class="MsoNormal"><span lang="EN-US">mutation or synthesis, we will generate a lot of garbage inputs which
</span></p>
<p class="MsoNormal"><span lang="EN-US">can only be used to detect crashes and hangs. However, given Alive we
</span></p>
<p class="MsoNormal"><span lang="EN-US">can verify if the output of the compiler is an implementation of the
</span></p>
<p class="MsoNormal"><span lang="EN-US">input, for some cases.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> To be honest, previous llvm passes I wrote are out tree passes. I’ve just setuped my machine, built llvm configured with fuzzer support, and started fiddling around lately. I have a rough picture of what each idea is
about, but it would take some preparation work for me to split them into incremental steps and deliverables. Since it’s still early in the application process, I wonder if you can spare me some time researching the ideas that you proposed and making inquiries
before finally deciding on my project proposal? 😊</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">As mentioned, students write the proposal. You should determine which of
</span></p>
<p class="MsoNormal"><span lang="EN-US">the "areas" you like best and then do some research towards that. We can
</span></p>
<p class="MsoNormal"><span lang="EN-US">be in contact and you start write up what you want to do.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> I am living in Shanghai, in the GMT+8 time zone. How about 15:00 tommorrow (March. 10), or 13:30 on Friday afternoon (March. 12)? I am not sure which time zone you are located in, so feel free to propose another time
slot if the prior two are not convenient for you (later that day or on weekends are both fine). Hope to have a chat with you soon.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">This week is full, I'll get back to you.</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">~ Johannes</span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> Cheers,</span></p>
<p class="MsoNormal"><span lang="EN-US">> Chibin Zhang</span></p>
<p class="MsoNormal"><span lang="EN-US">> 2021.3.9</span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> </span>发件人<span lang="EN-US">: Johannes Doerfert<mailto:johannesdoerfert@gmail.com></span></p>
<p class="MsoNormal"><span lang="EN-US">> </span>发送时间<span lang="EN-US">: 2021</span>年<span lang="EN-US">3</span>月<span lang="EN-US">9</span>日<span lang="EN-US"> 7:17</span></p>
<p class="MsoNormal"><span lang="EN-US">> </span>收件人<span lang="EN-US">: Florian Hahn<mailto:florian_hahn@apple.com>; llvm-dev<mailto:llvm-dev@lists.llvm.org></span></p>
<p class="MsoNormal"><span lang="EN-US">> </span>抄送<span lang="EN-US">: </span>张驰斌<span lang="EN-US"><mailto:zhangchb1@shanghaitech.edu.cn>; John Regehr<mailto:regehr@cs.utah.edu></span></p>
<p class="MsoNormal"><span lang="EN-US">> </span>主题<span lang="EN-US">: Re: [llvm-dev] Applying for GSoC 2021(Fuzzing LLVM-IR Passes)</span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> Having Alive2 as oracle would certainly be great.</span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> Some rough ideas that can be worked on in parallel if we have multiple</span></p>
<p class="MsoNormal"><span lang="EN-US">> GSoC students:</span></p>
<p class="MsoNormal"><span lang="EN-US">> - mutation rules we know are sound, e.g., remove guarantees, add 1</span></p>
<p class="MsoNormal"><span lang="EN-US">> iteration loops, etc.</span></p>
<p class="MsoNormal"><span lang="EN-US">> - input generation, equivalence checking (alive, partial evaluation, ...)</span></p>
<p class="MsoNormal"><span lang="EN-US">> - fragment extraction from larger codes + input tracking -></span></p>
<p class="MsoNormal"><span lang="EN-US">> reproducer splitting, faster equivalence checking, ...</span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> We certainly can come up with more things.</span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> Would either or both of your (or anyone else) be interested in</span></p>
<p class="MsoNormal"><span lang="EN-US">> co-mentoring students?</span></p>
<p class="MsoNormal"><span lang="EN-US">> We have multiple interested ones already, even though my project</span></p>
<p class="MsoNormal"><span lang="EN-US">> description is lacking any detail.</span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> ~ Johannes</span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">> On 3/8/21 3:34 PM, Florian Hahn wrote:</span></p>
<p class="MsoNormal"><span lang="EN-US">>>> On Mar 8, 2021, at 20:26, John Regehr via llvm-dev <llvm-dev@lists.llvm.org> wrote:</span></p>
<p class="MsoNormal"><span lang="EN-US">>>><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">>>> Hi folks, an angle related to IR fuzzing that I would be happy to help out with is using Alive2 as a test oracle.</span></p>
<p class="MsoNormal"><span lang="EN-US">>>><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">>>> Using Alive2 incurs a set of problems (not all IR features supported, can be very slow) but has corresponding advantages (considers all inputs at once, handles UB gracefully).</span></p>
<p class="MsoNormal"><span lang="EN-US">>>><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">>> If anyone’s interested in combing LLVM’s libFuzzer & Alive2, I’ve put up https://reviews.llvm.org/D96654 which uses Alive2 to verify candidates generated by fuzzing. It works out quite well, but I think there’s lots
of potential to improve the ‘interestingness’ of the IR generated by libFuzzer.</span></p>
<p class="MsoNormal"><span lang="EN-US">>><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">>> Cheers,</span></p>
<p class="MsoNormal"><span lang="EN-US">>> Florian</span></p>
<p class="MsoNormal"><span lang="EN-US">>><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</body>
</html>