<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Feb 22, 2016 at 2:11 PM, Robinson, Paul <span dir="ltr"><<a href="mailto:Paul_Robinson@playstation.sony.com" target="_blank">Paul_Robinson@playstation.sony.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div><span class="">
<p class="MsoNormal" style="margin-left:.5in">Agreed - further, having compilation of the "same" source having different IDs is an anti-goal, it would mean non-reproducible/stable builds, which we certainly don't want.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</span><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hmm, that seems worthwhile.<u></u><u></u></span></p><span class="">
<p class="MsoNormal" style="margin-left:.5in"><br>
Ideally, the DWO ID wouldn't change if you change text in a comment, even, or the name of a macro (if you don't have macro debug info enabled - though currently it would not change even if you did have macro debug info enabled, which is unfortunate). That way,
an optimal build system would rebuild the .o/.dwo, notice that they are identical to the previous ones, and not rebuild the linked executable or the dwp (if any).<u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
</span><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Well, seems like the DWO ID wouldn't *need* to change even if you made moderate code changes (as long as the types and variables and function signatures and
set of template instantiations etc didn't change). As long as the .debug_addr *indexes* didn't change, for example, it doesn't matter if the *content* of the .debug_addr section (in the .o file) did change. A hash of the content of the *.dwo sections would
DTRT here.</span></p></div></div></blockquote><div><br></div><div>Right<br><br>I think at the time we felt we had a circular problem when it came to hashing the entirety of the .dwo sections - no easy way in the MC layers to go & patch up the DWO ID into the .dwo sections you just hashed, etc. But I don't think that was the only problem - not sure, though.<br><br>- Dave</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">--paulr<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><a name="244636924__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></a></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> David Blaikie [mailto:<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>]
<br>
<b>Sent:</b> Sunday, February 21, 2016 4:42 PM<br>
<b>To:</b> Eric Christopher<br>
<b>Cc:</b> Robinson, Paul; Peter Collingbourne; Adrian Prantl; <a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a></span></p><div><div class="h5"><br>
<b>Subject:</b> Re: [PATCH] D17321: DIEData, DIEWriter: introduce and begin migration.<u></u><u></u></div></div><p></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Fri, Feb 19, 2016 at 7:06 PM, Eric Christopher via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Fri, Feb 19, 2016 at 6:52 PM Robinson, Paul <<a href="mailto:Paul_Robinson@playstation.sony.com" target="_blank">Paul_Robinson@playstation.sony.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Type units are COMDATs produced across (potentially) many compilations, so the key has to correspond
to the content in a predictable/consistent way in order for the linker to throw away duplicates. Therefore the type-unit hash is specified the way it is.</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Doesn't matter. The way that we're doing it in clang is perfectly acceptable for this.<u></u><u></u></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Ish - there are tradeoffs for sure. The way we're doing this in Clang means that if the user violates the ODR they get a weirder experience in the debugger than not. But we decided to make that tradeoff to simplify/optimize/etc the implementation.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">dwo_ids are just identifiers matching two lumps of data for the same CU, produced by the same compilation,
so they aren't sensitive to content (at least not in the same way as type units). But it would seem prudent for two compilations of the "same" source to have different IDs, thus hashing the .debug_info content might not be the best choice; a more random-number
kind of ID would be more appropriate.</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I don't see this either, but I know where you're coming from.<u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Agreed - further, having compilation of the "same" source having different IDs is an anti-goal, it would mean non-reproducible/stable builds, which we certainly don't want.<br>
<br>
Ideally, the DWO ID wouldn't change if you change text in a comment, even, or the name of a macro (if you don't have macro debug info enabled - though currently it would not change even if you did have macro debug info enabled, which is unfortunate). That way,
an optimal build system would rebuild the .o/.dwo, notice that they are identical to the previous ones, and not rebuild the linked executable or the dwp (if any).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">-eric<u></u><u></u></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">--paulr</span><u></u><u></u></p>
<p class="MsoNormal"><a name="244636924_1942077568_msg-f:1526660323155250066__Ma"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></a><u></u><u></u></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> llvm-commits [mailto:<a href="mailto:llvm-commits-bounces@lists.llvm.org" target="_blank">llvm-commits-bounces@lists.llvm.org</a>]
<b>On Behalf Of </b>Eric Christopher via llvm-commits<br>
<b>Sent:</b> Friday, February 19, 2016 6:39 PM<br>
<b>To:</b> Peter Collingbourne; Adrian Prantl<br>
<b>Cc:</b> <a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<b>Subject:</b> Re: [PATCH] D17321: DIEData, DIEWriter: introduce and begin migration.</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
<div>
<div>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Fri, Feb 19, 2016 at 5:45 PM Peter Collingbourne via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal">On Fri, Feb 19, 2016 at 03:47:22PM -0800, Adrian Prantl wrote:<br>
><br>
> > On Feb 19, 2016, at 12:31 PM, Peter Collingbourne via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br>
> ><br>
> > them incorrectly, see DwarfDebug::makeTypeSignature). Since the page at<br>
> > <a href="https://gcc.gnu.org/wiki/DebugFission" target="_blank">https://gcc.gnu.org/wiki/DebugFission</a> does not specify how the hash is to be<br>
> > computed, could we maybe do something simpler (e.g. MD5 hash the DIE bytes<br>
> > themselves) and avoid needing to use something like the DWARF type hashing<br>
> > algorithm for this?<br>
><br>
> Where it can LLVM uses an MD5 sum over the mangled name of the type to avoid the expensive DWARF type hashing.<br>
><br>
> Just a side node: Debug info fission is in the process of being standardized in DWARF 5. The progress can be tracked here:<br>
> <a href="http://dwarfstd.org/Issues.php?type=closed4" target="_blank">http://dwarfstd.org/Issues.php?type=closed4</a> (search for “fission” and “split DWARF”)<br>
> The DWARF specification may deviate from the description on the GCC wiki a little, but we probably want to follow the DWARF spec where that makes sense.<br>
<br>
I see. From <a href="http://dwarfstd.org/ShowIssue.php?issue=130313.4" target="_blank">
http://dwarfstd.org/ShowIssue.php?issue=130313.4</a> :<br>
<br>
> 5. A DW_AT_dwo_id attribute whose value is an 8-byte<br>
> unsigned hash of the full compilation unit. This hash<br>
> value is computed by the method described in Section 7.27<br>
> ("Type Signature Computation").<br>
<br>
That's unfortunate. Maybe I'm missing something, but I don't really see why<br>
the attribute value needs to be specified like that, as the hash is only<br>
used by consumers to match skeleton units with full units, and it should be<br>
up to the producer to use a good enough hash. Is there any possibility of<br>
changing the specification before DWARF 5 is standardised?<u></u><u></u></p>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">It has been, but it's not really important as any two compilers aren't going to agree on debug info output anyhow so they're unlikely to match. I raised this in committee at the
time as a "may" rather than "is". That said, recent text (I don't have an absolutely updated version in front of me) reads:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">A DW_AT_dwo_id attribute whose implementation-defined integer constant value, known as the compilation unit ID, provides unique identification of this compilation unit as well as
the associated split compilation unit in the object file named in the DW_AT_dwo_name attribute. For simplicity, the DW_AT_dwo_id attributes in the skeleton compilation unit and the corresponding split full compilation unit (see Section 3.1.3 on the following
page) must use the same form to encode this identification value.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">The means of determining a compilation unit ID does not need to be similar or related to the means of determining a type unit signature.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">That said, ISTR at the time having a good reason why a hash of the debug_info section wasn't a sufficient method for unique identifiers, but my brain isn't coming up with it right
now. Might have been related to not having a finalized section before wanting to splat in the hash, but I'm not sure.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">For the type ids we haven't worried because we're only supporting type units for C++ types at the moment, if we wanted to support C types we'd want to come up with another identifier
so we could merge similar types across compilation units.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Hope this helps.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-eric<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></div></div>
</div>
</div>
</blockquote></div><br></div></div>