<div dir="ltr"><div dir="ltr">On Mon, Jul 26, 2021 at 12:58 PM Alexander Yermolovich <<a href="mailto:ayermolo@fb.com">ayermolo@fb.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
Haven't seen overflows in Split DWARF yet</div></div></blockquote><div><br>Careful, as they're totally silent (at least with gold dwp, and probably also with llvm dwp) - the str_offsets get overflowed values, and then when the data is read by the DWARF consumer, the strings end up corrupted - because you're reading from arbitrary/incorrect offsets.<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">, but thanks for letting me know, and the links to discussions. Is there a plan to productize either one or both?<br></div></div></blockquote><div><br>Yep, the plan on both counts is to upstream them. I have the simplified template names implementation on the go at the moment - adding a flag to clang that implements the functionality, but also implements a "mangled" mode, where if a name should eb able to be simplified instead it's emitted in full with a special prefix ("_STN") - and then the consumer can attempt to reconstitute that name and compare it against the name provided (& the llvm-dwarfdump --verify mode does this checking and fails if they don't match). So I'm going through lots of cases, either adding the rebuilding logic that's needed, or modifying the frontend not to simplify/mark certain names that can't be rebuilt.<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">For us, in monolithic format, it was .debug_info that was growing too large and relocations failing in to, or out of it. The.debug_aranges relocations in to it, and don't quite remember from top of my head what out relocation was in to. I think it was .debug_loc<br></div></div></blockquote><div><br>Huh, fascinating. Good to know!<br><br>- Dave<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<br>
Alex</div>
<div id="gmail-m_-2472033645595274120appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-2472033645595274120divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>><br>
<b>Sent:</b> Friday, July 23, 2021 11:58 AM<br>
<b>To:</b> Alexander Yermolovich <<a href="mailto:ayermolo@fb.com" target="_blank">ayermolo@fb.com</a>><br>
<b>Cc:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a> <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] [RFC][Dwarf Library] Relocations for DWO sections</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div dir="ltr">On Fri, Jul 23, 2021 at 1:18 PM Alexander Yermolovich <<a href="mailto:ayermolo@fb.com" target="_blank">ayermolo@fb.com</a>> wrote:<br>
</div>
<div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
Thanks for replying in the patch. Left my reply.<br>
We are using it to deal with dwarf relocation overflows. </div>
</div>
</blockquote>
<div><br>
Ah, that's good to know. FWIW we've started to hit some overflows even in Split DWARF on larger binaries (and/or those making especially heavy use of expression templates - creating an exceptional amount of DWARF/long symbol names <br>
<br>
A couple of ideas to address this particular overflow (which section(s) did you manage to overflow? We're dealing with .debug_str[.dwo] overflow in particular) that I'm looking into are:<br>
Simplified template names ( <a href="https://lists.llvm.org/pipermail/llvm-dev/2021-June/150903.html" target="_blank">https://lists.llvm.org/pipermail/llvm-dev/2021-June/150903.html</a>
) - emit only the base name ("foo") of a template rather than all the template parameters ("foo<int>") - and then reconstruct the full name by using the DW_TAG_template_type_parameters, etc.<br>
Reconstituted Mangled names ( <a href="https://groups.google.com/g/llvm-dev/c/2jMqDjdChuQ/m/HpOpWy8pAwAJ" target="_blank">https://groups.google.com/g/llvm-dev/c/2jMqDjdChuQ/m/HpOpWy8pAwAJ</a>
) - skip mangled names when they can be reconstituted from the DWARF structural representation (eg: "void f1(int) { }" -> "_Z2f1i" but we could build the latter from DWARF's representation that says f1 has one "int" parameter).<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
We considered DWARF64, but split dwarf seems like a more traveled path. As for single vs split my understanding is that single plays nicer with our build system ATM.</div>
</div>
</blockquote>
<div><br>
Ah, fair enough.<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div id="gmail-m_-2472033645595274120x_gmail-m_-1700662945782993288appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-2472033645595274120x_gmail-m_-1700662945782993288divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>><br>
<b>Sent:</b> Friday, July 23, 2021 7:41 AM<br>
<b>To:</b> Alexander Yermolovich <<a href="mailto:ayermolo@fb.com" target="_blank">ayermolo@fb.com</a>><br>
<b>Cc:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a> <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] [RFC][Dwarf Library] Relocations for DWO sections</font>
<div> </div>
</div>
<div>
<div dir="ltr">General premise sounds correct to me (that we shouldn't be processing those sections, etc). I've replied to the patch - thanks for taking a look at this!<br>
<br>
(out of curiosity: What are you using Split DWARF single mode for (if you can speak to the application)?)</div>
<br>
<div>
<div dir="ltr">On Thu, Jul 22, 2021 at 9:10 PM Alexander Yermolovich via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
Hello<br>
<br>
I observed when DWARF Context is created for DWO object (split dwarf single mode), that relocations for .debug_info are processed and are stored in a map. This adds quite a bit of memory overhead. This doesn't seem like it is needed for DWO Context. Context
created through API DWARFContext::getDWOContext. Am I missing something?<br>
<br>
Illustrative patch to fix this:<br>
<a href="https://reviews.llvm.org/D106624" id="gmail-m_-2472033645595274120x_gmail-m_-1700662945782993288x_gmail-m_1795294339709795769LPlnk172681" target="_blank">https://reviews.llvm.org/D106624</a><br>
<br>
Thank you,
<div>Alex</div>
</div>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote></div></div>