<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Here's a broader precis of my thinking in taking the library in this direction.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Prior to v4, DWARF kept information of different natures in different object-file sections. So, .debug_line had the line tables, .debug_macinfo had the macro
information, .debug_info had the DIEs, and so on. In some cases, information in a single section was subdivided according to compilation unit (.debug_info and .debug_line in particular) but the information in each section was all of the same nature.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">In v4, DWARF added .debug_types, which was different in two ways. First, there might be multiple object-file sections with the same name (the COMDATs), and
second, the nature of the information was much the same as in .debug_info. There was a distinction, however, in that .debug_info contained only compile units and .debug_types contained only type units. There's also the aspect that the LLVM parser is oriented
toward the needs of an object-file dumper, which encouraged thinking about the information in object-file terms, i.e. by section. Hence, type units were supported by using a container of section-oriented classes, each element being able to handle multiple
units.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">In v5, DWARF corrected the mistake of putting type units in a differently named object-file section. So, now we can have one or more .debug_info sections,
each of which can contain one or more compile and/or type units. But of course .debug_types still exists if there is some linked object file containing a mix of v4 and 5 compilations.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">In parallel, there is increased interest in making the LLVM and LLDB parsers converge, and if the decision goes with using the LLVM parser in LLDB, the LLVM
parser needs to better accommodate the needs of a debugger rather than just a dumper. In that case, the important distinction is not the object-file section, but the nature of the information. That is, pasting together all the DIE data from .debug_info and
.debug_types into a single manageable whole is what LLDB wants, because the source section is irrelevant. And in fact, within the past year LLDB's parser was modified to understand .debug_types, and models it as being appended to .debug_info for navigation
purposes.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Of course the LLVM dumper still thinks about things in object-file section terms, so it's worth being able to look at the DIE information on a per-section basis.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I think I have achieved that with the refactoring. The DWARF data that is in the nature of DIEs is managed by one container, making it easier for LLDB to look
at it as a unified whole. The container can provide iterators that distinguish between different section names, allowing the dumper to retain a section-oriented view. The functional patch that enables multiple .debug_info sections for DWARF v5 should be
not a whole lot more than changing the parse methods to iterate over possibly multiple instances of object-file sections with that name.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">If there are bugs in how intra-section offsets are reported and handled, then it's worth knowing; what that means is that we don't have existing tests that
demonstrate the offsets are done correctly. Happy to improve testing and fix any problems found.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">If there is bikeshedding about the names of methods, that's fine too. Renaming DWARFUnitVector::parse[DWO] to parse[DWO]Section is obviously simple and can
be folded into the same patch that renamed the class.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">As for merging DWARFTypeUnit and DWARFCompileUnit into DWARFUnit and eliminating the distinction at the class level, I haven't really looked at that. There
are a couple reasons to put that off. First, Jan Kratochvil has been doing LLDB work in the direction of more subclasses, not fewer, i.e. putting partial and imported units into their own subclasses. I haven't looked closely at his patches. Second, I've
had interest internally in getting partial units to work reasonably so we can put them into COMDATs and trivially deduplicate the debug info for linkonce functions. If a separate subclass would help there, merging others would make less sense.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">So I'm not fundamentally opposed to merging the unit classes, but I think it can be deferred with little problem while we work out the other aspects.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Other questions/comments always welcome.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">--paulr<o:p></o:p></span></p>
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></a></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> David Blaikie [mailto:dblaikie@gmail.com]
<br>
<b>Sent:</b> Wednesday, July 25, 2018 3:36 PM<br>
<b>To:</b> Robinson, Paul<br>
<b>Cc:</b> llvm-dev@lists.llvm.org<br>
<b>Subject:</b> Re: [llvm-dev] [DWARF] De-segregating type units and compile units<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">A few general design discussions that cross multiple patches, so figure it's worth discussing here:<br>
<br>
The DWARFUnitSection is currently (before your changes) representing the contents from a single section - and most/all of the classes in libDebugInfoDWARF use "parse" to initialize their complete state. So it's a bit novel/new/different/worth considering (maybe
renaming the function or the like) what it means to make this class now represent data from multiple sections and to be documentend/intended to be used to aggregate the parsing result from multiple sections (both multiple sections with the same name (debug_types
currently, or in v5 debug_info comdat sections for type units)). One observable part of the matter of which sections come from is the section relative offsets (you'll see the difference in dumping a file with at least 2 types with type units enabled, with
and without split DWARF - without split DWARF comdat sections are used, so each types section offsets are zerod at the start of each type unit because it's a separate section (even though it has the same section name) - whereas in split DWARF, in the .dwo
file the offsets are across the whole, singular, debug_types.dwo section)<br>
<br>
As for virtuality in the unit hierarchy - reckon it's worth it, or maybe just use a union/conditional for the dumping differences (& collapse/remove the type distinction between type units and compile units - just have DWARFUnit as the only unit class)? (since
it's just the type_signature V DWO_id (maybe we could generalize that to "id") and type_offset that's different between them).<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Jul 24, 2018 at 10:40 AM via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">Hello DWARF fans,<br>
<br>
I've just posted a set of four refactoring patches for DebugInfo/DWARF,<br>
which move in the direction of handling DWARF v4 or v5 type units and<br>
compile units more coherently.<br>
<br>
In DWARF v4, type units and compile units are strictly segregated into<br>
the .debug_types and .debug_info sections, respectively. This division<br>
was pretty ingrained into how DebugInfo/DWARF handled the units.<br>
<br>
In DWARF v5, type units and compile units are all in the .debug_info<br>
section, and the .debug_types section is obsolete. So, we need to have<br>
a container than can handle both kinds of units in a straightforward way.<br>
<br>
The refactoring replaces a pair of collections templated on the unit type<br>
with a single collection whose elements are base-class pointers; thus it<br>
can contain elements that are a mix of unit kinds. Everything that really<br>
mattered was either already virtual or was straightforward to convert to<br>
generic handling. I think I needed only one type-based conditional, on<br>
top of what already existed. The code doesn't *quite* support DWARF v5<br>
type units in the .debug_info section, but the new starting point should<br>
make that straightforward.<br>
<br>
In a mixed v4/v5 executable, we can pretend the .debug_types sections<br>
are all really .debug_info sections, and tack them onto the end of the<br>
vector of units that already handles a mix of compile and type units.<br>
This is also how the LLDB DWARF parser handles this case, so it's at<br>
least consistent in principle (even if the actual code differs).<br>
<br>
The existing distinction between "normal" and split (DWO) units remains;<br>
that has to do with what information exists in which object files, and<br>
is not fundamentally changing in DWARF v5.<br>
<br>
Patch 1: De-templatize DWARFUnitSection<br>
<a href="https://reviews.llvm.org/D49741" target="_blank">https://reviews.llvm.org/D49741</a><br>
Patch 2: The TU collection doesn't need to be a deque<br>
<a href="https://reviews.llvm.org/D49742" target="_blank">https://reviews.llvm.org/D49742</a><br>
Patch 3: Rename DWARFUnitSection to DWARFUnitVector<br>
<a href="https://reviews.llvm.org/D49743" target="_blank">https://reviews.llvm.org/D49743</a><br>
Patch 4: Unify handling of type and compile units<br>
<a href="https://reviews.llvm.org/D49744" target="_blank">https://reviews.llvm.org/D49744</a><br>
<br>
Thanks,<br>
--paulr<br>
<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</body>
</html>