<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Apr 1, 2016 at 2:19 AM, Aboud, Amjad via llvm-dev <span dir="ltr"><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">I will say it one more time, it does not sound a good design to change the IR according to the target debug info format.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Why?<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">This breaks the modularity of LLVM compiler, assume we want to support another debug info format in the future? Do you expect
us to add another Record/Type specification to the IR?<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">This means that we need to modify Clang FE, LLVM MED optimizations, and maybe duplicate/rewrite all related LIT tests , in addition
to the expected work in codegen.</span></p></div></div></blockquote><div><br></div><div>We don't expect to need to modify the middle end optimizations - type information is generally opaque to them.<br><br>This actually reduces the test surface - as we don't have to test generation of two different flavors of IR metadata as well as two different flavors of final output. The output is produced only once, in the frontend. (which we expect to need to modify for new debug info formats with different features anyway) & some work in the backend (which needs wholely new support for any new debug info format as well)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">As I see it, we can easily separate the FE/MED parts from backend (codegen).<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">In FE/MED we should have one representation for all debug info that capture the debug information from the sources (we should
not lose information just because some formats, like DWARF, does not need them!)</span></p></div></div></blockquote><div>For types in particular, this information is essentially equivalent to ASTs - it's a lot to generalize over & having a format that can represent them all (or has the complication of unions over different subsets so as not to pay the cost of the sum of all debug info format properties) is somewhat costly. It adds a lot of work to funnel this data through what isn't really a very general system. Every time we add a new debug info feature (as you saw with using directives, for example) it's a lot of plumbing through many layers. The less stuff that has to have a new/separate representation through all those layers, the better. <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">In Backend, we should have separate emitters that convert the debug information captured in the IR into suitable data structures
that are related to the target debug info format.<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">In addition, if we think that some information will be much easier to calculate in FE rather than BE, we can create it either always,
or according to the target, but in this case it should be defined as optional field in the debug info IR rather than a totally new separate debug info IR.</span></p></div></div></blockquote><div><br></div><div>That's partly the issue - the features necessary for the Windows ABI's debug info in PDB files aren't generic, they're pretty specific, need to be computed in the frontend and can't really be optional (any more than debug info in general is optional).<br><br>- Dave</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal" style="margin-left:.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Amjad<u></u><u></u></span></p>
<p class="MsoNormal"><a name="m_4081904575973794358__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></a></p>
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><a name="m_4081904575973794358______replyseparator"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Reid Kleckner [mailto:<a href="mailto:rnk@google.com" target="_blank">rnk@google.com</a>]
<br>
<b>Sent:</b> Friday, April 01, 2016 01:47<br>
<b>To:</b> Aboud, Amjad <<a href="mailto:amjad.aboud@intel.com" target="_blank">amjad.aboud@intel.com</a>><br>
<b>Cc:</b> <a href="mailto:mehdi.amini@apple.com" target="_blank">mehdi.amini@apple.com</a>; llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>; Clang Dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] [cfe-dev] RFC: Up front type information generation in clang and llvm<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">The split between CodeView and DWARF will happen at the level of type information. So, DIVariable, DISubprogram, DILocation, DILocalScope, etc will all be shared, but records and composite types etc will not.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Thu, Mar 31, 2016 at 3:44 PM, Aboud, Amjad via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Hi Mehdi,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">I understand the reasoning for supporting this proposal independently from CodeView support.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">However, I do not think that it is needed for supporting CodeView.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">When I say that my suggestion is more clean, I was pointing to CodeView support, assuming the changes
in LLVM IR/Clang FE indicated in this proposal.</span><u></u><u></u></p>
<p class="MsoNormal"><a name="m_4081904575973794358_m_-7991684042482456353__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Also, it is not that clear from the proposal what
will be shared (generic) between Dwarf and CodeView and what will be specific.</span></a><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Regards,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Amjad</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><a name="m_4081904575973794358_m_-7991684042482456353______replyseparat"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">
<a href="mailto:mehdi.amini@apple.com" target="_blank">mehdi.amini@apple.com</a> [mailto:<a href="mailto:mehdi.amini@apple.com" target="_blank">mehdi.amini@apple.com</a>]
<br>
<b>Sent:</b> Thursday, March 31, 2016 22:27<br>
<b>To:</b> Aboud, Amjad <<a href="mailto:amjad.aboud@intel.com" target="_blank">amjad.aboud@intel.com</a>><br>
<b>Cc:</b> Eric Christopher <<a href="mailto:echristo@gmail.com" target="_blank">echristo@gmail.com</a>>; Clang Dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>>; llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [cfe-dev] [llvm-dev] RFC: Up front type information generation in clang and llvm</span><u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Hi Aboud,<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Mar 31, 2016, at 11:06 AM, Aboud, Amjad via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Hi Eric,</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">I can understand the need for improving the current design of debug info representation and emission
in LLVM.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">However, let’s not forget that the motivation was and still to support CodeView debug info emission.</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Well, that is *one* motivation.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">I am wondering if it is right to spend the huge effort needed to implement the below proposal while
knowing these facts:</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">1.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">It
would be more clear how to improve the design when we have a working CodeView support.</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">You said it yourself, that we still do not know what challenges we will face while implementing this
proposal.</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">2.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">I
understand that CodeView will need some extra extensions to current dwarf debug info, like ‘this’ adjustment.</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">However, it is doable to introduce a CodeView wrapper data structures that can be created from current
dwarf debug info IR.</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">And this can be done in CodeGen (e.g. CodeViewDebug.cpp) while emitting the code/debug info.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Again, I understand that your proposal is trying to improve a lot of things</span><u></u><u></u></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Yes, and to give some different perspective: some of these "things" are a lot higher priority than CodeView (for other people/use cases of course), because DebugInfo cost is prohibitive
for some use cases.<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">, but it seems that we should first try support CodeView debug info with the current debug info IR.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">The advantages:</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">1.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">It
works, even though you still have doubts about few issues, I believe we can resolve them with minimum modification to the LLVM IR/Clang FE.</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">2.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">It
requires much smaller effort.</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">3.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">It
is much clean.</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">If it is "much cleaner" in the IR, I understand that you have insights about Eric's proposal being "less clean", independently of adding CodeView before or after this change. If
so it's worth elaborating on this.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">4.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">We
will understand more the requirements needed by CodeView that can be used to improve the below proposal (before diving into implementing it).</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">Don't you forget the "Cons":<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">1) It is easier to perform large refactoring/changes to the debug info flow *before* complexifying the problem.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2) This is adding more stuff that will need to go through all these changes, wasting effort in the process.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">3) It will limit forward progress for people who don't care about CodeView but want to move forward with restructuring DI deeply, like Eric's proposal is doing.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">That is not to say that your points are not valid, but that it's not that clear cut either.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">-- <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Mehdi<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">I suggest that we start with:</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">1.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Define
the CodeView wrapper data structure. (CodeViewDebugIR)</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">2.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Build
the CodeView wrapper data structure based on dwarf debug info IR. (CodeViewDebugBuilder)</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">3.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Emit
the CodeView wrapper data structure into COFF object file. (CodeViewDebugEmitter)</span><u></u><u></u></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">4.</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Figure
out what modification/extension need to be done to dwarf debug info IR/Clang FE.</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">What do you think?</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Thanks,</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Amjad</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span><u></u><u></u></p>
</div>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<div>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> llvm-dev [<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank"><span style="color:#954f72">mailto:llvm-dev-bounces@lists.llvm.org</span></a>] <b>On
Behalf Of </b>Eric Christopher via llvm-dev<br>
<b>Sent:</b> Wednesday, March 30, 2016 04:01<br>
<b>To:</b> Clang Dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank"><span style="color:#954f72">cfe-dev@lists.llvm.org</span></a>>; llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank"><span style="color:#954f72">llvm-dev@lists.llvm.org</span></a>><br>
<b>Subject:</b> [llvm-dev] RFC: Up front type information generation in clang and llvm</span><u></u><u></u></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Hi All,<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">This is something that's been talked about for some time and it's probably time to propose it.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">The "We" in this document is everyone on the cc line plus me.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Please go ahead and take a look.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Thanks!<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">-eric<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma",sans-serif"></span>Objective (and TL;DR)<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">=================<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Migrate debug type information generation from the backends to the front end.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">This will enable:<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">1. Separation of concerns and maintainability: LLVM shouldn’t have to know about C preprocessor macros, Obj-C properties, or extensive details about debug information binary formats.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">2. Performance: Skipping a serialization should speed up normal compilations.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">3. Memory usage: The DI metadata structures are smaller than they were, but are still fairly large and pointer heavy.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Motivation<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">========<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Currently, types in LLVM debug info are described by the DIType class hierarchy. This hierarchy evolved organically from a more flexible sea-of-nodes representation into what it
is today - a large, only somewhat format neutral representation of debug types. Making this more format neutral will only increase the memory use - and for no reason as type information is static (or nearly so). Debug formats already have a memory efficient
serialization, their own binary format so we should support a front end emitting type information with sufficient representation to allow the backend to emit debug information based on the more normal IR features: functions, scopes, variables, etc.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Scope/Impact<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">===========<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">This is going to involve large scale changes across both LLVM and clang. This will also affect any out-of-tree front ends, however, we expect the impact to be on the order of a
large API change rather than needing massive infrastructure changes.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Related work<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">==========<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">This is related to the efforts to support CodeView in LLVM and clang as well as efforts to reduce overall memory consumption when compiling with debug information enabled; in particular
efforts to prune LTO memory usage.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Concerns<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">========<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">We need a good story for transitioning all the debug info testcases in the backend without giving up coverage and/or readability. David believes he has a plan here.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Proposal<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">=======<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Short version<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">-----------------<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">1. Split the DIBuilder API into Types (+Macros, Imports, …) and Line Table.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">2. Split the clang CGDebugInfo API into Types and Line Table to match.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">3. Add a LLVM DWARF emission library similar to the existing CodeView one.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">4. Migrate the Types API into a clang internal API taking clang AST structures and use the LLVM binary emission libraries to produce type information.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">5. Remove the old binary emission out of LLVM.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Questions/Thoughts/Elaboration<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">-------------------------------------------<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Splitting the DIBuilder API<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">~~~~~~~~~~~~~~~~~~~~<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Will DISubprogram be part of both?<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * We should split it in two: Full declarations with type and a slimmed down version with an abstract origin.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">How will we reference types in the DWARF blob?<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * ODR types can be referenced by name<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * Non-odr types by full DWARF hash<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * Each type can be a pair(tuple) of identifier (DITypeRef today) and blob.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * For < DWARF4 we can emit each type as a unit, but not a DWARF Type Unit and use references and module relocations for the offsets. (See below)<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">How will we handle references in DWARF2 or global relocations for non-type template parameters?<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * We can use a “relocation” metadata as part of the format.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * Representable as a tuple that has the DIType and the offset within the DIBlob as where to write the final relocation/offset for the reference at emission time.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Why break up the types at all?<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * To enable non-debug format aware linking and type uniquing for LTO that won’t be huge in size. We break up the types so we don’t need to parse debug information to link two
modules together efficiently.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Any other concerns there?<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * Debug information without type units might be slightly larger in this scheme due to parents being duplicated (declarations and abstract origin, not full parents). It may be
possible to extend dsymutil/etc to merge all siblings into a common parent. Open question for better ways to solve this.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">How should we handle DWARF5/Apple Accelerator Tables?<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * Thoughts:<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * We can parse the dwarf in the back end and generate them.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * We can emit in the front end for the base case of non-LTO (with help from the backend for relocation aspects).<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * We can use dsymutil on LTO debug information to generate them.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Why isn’t this a more detailed spec?<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> * Mostly because we’ve thought about the issues, but we can’t plan for everything during implementation.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Future work<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">----------------<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Not contained as part of this, but an obvious future direction is that the Module linker could grow support for debug aware linking. Then we can have all of the type information
for a single translation unit in a single blob and use the debug aware linking to handle merging types.<u></u><u></u></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="text-align:start;word-spacing:0px">
<span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">---------------------------------------------------------------------<br>
Intel Israel (74) Limited</span><u></u><u></u></p>
<p class="MsoNormal" style="text-align:start;word-spacing:0px">
<span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">This e-mail and any attachments may contain confidential material for<br>
the sole use of the intended recipient(s). Any review or distribution<br>
by others is strictly prohibited. If you are not the intended<br>
recipient, please contact the sender and delete all copies.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">_______________________________________________<br>
cfe-dev mailing list<br>
</span><a href="mailto:cfe-dev@lists.llvm.org" target="_blank"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif;color:#954f72">cfe-dev@lists.llvm.org</span></a><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><br>
</span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" target="_blank"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif;color:#954f72">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</span></a><u></u><u></u></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
<div>
<div>
<p>---------------------------------------------------------------------<br>
Intel Israel (74) Limited<u></u><u></u></p>
<p>This e-mail and any attachments may contain confidential material for<br>
the sole use of the intended recipient(s). Any review or distribution<br>
by others is strictly prohibited. If you are not the intended<br>
recipient, please contact the sender and delete all copies.<u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><u></u><u></u></p>
</blockquote>
</div>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</div></div></div><div><div class="h5">
<p>---------------------------------------------------------------------<br>
Intel Israel (74) Limited</p>
<p>This e-mail and any attachments may contain confidential material for<br>
the sole use of the intended recipient(s). Any review or distribution<br>
by others is strictly prohibited. If you are not the intended<br>
recipient, please contact the sender and delete all copies.</p></div></div></div>
<br>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
<br></blockquote></div><br></div></div>