<div dir="ltr">Ping.<div><br></div><div>Still working on preparing code for review. Will have a patch for review ready in the coming days. </div><div><br></div><div>PL</div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Aug 15, 2017 at 12:06 PM Puyan Lotfi <<a href="mailto:puyan.lotfi.llvm@gmail.com">puyan.lotfi.llvm@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span id="m_2452468905233419516gmail-docs-internal-guid-44256f1b-e740-950e-75a8-4282e27328f5"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><span style="font-size:14.666666984558105px;font-weight:normal">Hi,</span></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><span style="font-size:14.666666984558105px;font-weight:normal"><br></span></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><span style="font-size:14.666666984558105px;font-weight:normal">My name is Puyan and </span></span><span style="color:rgb(0,0,0);font-family:Arial;font-size:14.666666984558105px;white-space:pre-wrap;background-color:transparent">I've been exploring ways to improve the state of instruction level diffing using llvm and MIR. Below is a proposal for a new llvm tool meant to address issues encountered when diffing at the machine level. I'm eager to hear the community's feedback.</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);font-family:Arial;font-size:14.666666984558105px;white-space:pre-wrap;background-color:transparent"><br></span></p><p style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);font-family:Arial;font-size:14.666666984558105px;white-space:pre-wrap;background-color:transparent">Thanks</span></p><p style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);font-family:Arial;font-size:14.666666984558105px;white-space:pre-wrap;background-color:transparent"><br></span></p><p style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);font-family:Arial;font-size:14.666666984558105px;white-space:pre-wrap;background-color:transparent">PL</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">mir-canon: A new tool for canonicalizing MIR for cleaner diffing.</span></p><br><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Problem Statement and Context:</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Diff-tools are regularly used for comparing IR and assembly files. This often involves reasoning through differences that are semantically equivalent and it can be very time consuming for a person to do said reasoning.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Specifically in the context of GlobalISel development there are problems of correctness verification. There is a need to compare two programs, compiled from identical IR by two different instruction selectors in a way where the true differences stand out.</span></p><br><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Proposal:</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">We propose a new tool that we have tentatively named 'mir-canon' that performs canonical transformations on MIR. The goal is for MIR pre-processed with mir-canon to show fewer differences than if it were not pre-processed.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">At the time of this writing we have a prototype canonicalization tool. We’ve come up with some techniques that show promise and would like to open discussion with the community to get feedback and suggestions on refining said techniques. Currently we think of this as an open ended project.</span></p><br><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Techniques:</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Our prototype does the following for each basic block in a Reverse Post Ordering:</span></p><br><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Canonical instruction ordering is done by moving a given instruction as close to the nearest use of its def as possible.</span></p></li></ul><br><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Next, canonical VReg renaming is done by building a collection of candidate instructions that can be thought of as sinks in the def-use graph: they are typically instructions that write to physical registers or store to memory. These candidates are used as the root of a breadth first walk over the vreg operand def-use graph that determines a canonical vreg ordering.</span></p></li></ul><br><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Using said canonical vreg ordering we rename monotonically, but before we do this we skip several vreg values in order to increase the chance that we land on the same vreg number for two different input MIR files. We also do this to reduce the chances that a difference in previously def-use walks will affect the vreg renaming for subsequent walks. This skipping step could be thought of as a kind of vreg number reckoning: we skip modulo n vregs so that we are likely to land on the same vreg for two different files.</span></p></li></ul><br><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">This approach is completely agnostic of ISA specific semantics so it should work for any target.</span></p><br><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Current status:</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">At the moment we have a standalone llvm tool that uses a single pass to do the above described transformations. We have test inputs that show promise but we still need a wider set of tests as well as hard metrics.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Our approach processes a single file at a time. The primary benefit to this approach is lower complexity in initial implementation and exploration of building such a tool. We are open to other approaches such as an llvm-diff like (two file at a time) approach, but we have not explored that avenue fully yet.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">We’re eager to hear community feedback and will be ready to share patches for review soon.</span></p><div><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-variant-ligatures:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br></span></div></span></div>
</blockquote></div>