<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Sep 1, 2020 at 10:18 AM James Y Knight <<a href="mailto:jyknight@google.com">jyknight@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 31, 2020 at 11:06 AM Alexey <<a href="mailto:avl.lapshin@gmail.com" target="_blank">avl.lapshin@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
Hi James,<br>
<br>
Thank you for the comments. <br>
<br>
>I think we're not terribly far from that ideal, now, for ELF.
Maybe only these three things need to be done? --<br>
> 1. Teach lld how to emit a separated debuginfo output file
directly, without requiring an objcopy step.<br>
> 2. Integrate DWARFLinker into lld.<br>
> 3. Create a new tool which takes the separated debuginfo and
DWO/DWP files and uses DWARFLinker library <br>
> to create a new (dwarf-linked) separated-debug file, that
doesn't depend on DWO/DWP files.<br>
<br>
The three goals which you`ve described are our far goals. <br>
Indeed, the best solution would be to create valid optimized debug
info without additional <br>
stages and additional modifications of resulting binaries. <br>
<br>
There was an attempt to use DWARFLinker from the lld -
<a href="https://reviews.llvm.org/D74169" target="_blank">https://reviews.llvm.org/D74169</a><br>
It did not receive enough support to be integrated yet. There are
fair reasons for that:<br>
<br>
1. Execution time. The time required by DWARFLinker for processing
clang binary is 8x bigger<br>
than the usual linking time. Linking clang binary with DWARFLinker
takes 72 sec, <br>
linking with the only lld takes 9 sec.<br>
<br>
2. "Removing obsolete debug info" could not be switched off. Thus,
lld could not use DWARFLinker for<br>
other tasks(like generation of index tables - .gdb_index,
.debug_names) without significant performance <br>
degradation.<br>
<br>
3. DWARFLinker does not support split dwarf at the moment.<br>
<br>
All these reasons are not blockers. And I believe implementation
from D74169 might be integrated and <br>
incrementally improved if there would be agreement on that.</div></blockquote><div><br></div><div>Those do sound like absolutely critical issues for deploying this for real -- whether as a separate tool or integrated with lld. But possibly not critical enough to prevent adding this behind an experimental flag, and working on the code incrementally in-tree. However (without having looked at the code in question), </div></div></div></blockquote><div><br></div><div>Yep, that's my feeling too.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>I wonder if the reported 8x regression in link-time is even going to be salvageable just by incremental optimizations, or if it might require a complete re-architecting of the DwarfLinker code.</div></div></div></blockquote><div><br>Jonas, who's looked at llvm-dsymutil performance for its own sake (motivated to improve llvm-dsymutil runtime, etc) & has mentioned on this/related threads that there might be minimal headroom to improve things there - so, yes, if there are greater opportunities it may require a fairly large/broad investment (though a second/third set of eyes on the current code to see if there are some hidden opportunities isn't a bad thing).<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>Using DWARFLinker from llvm-dwarfutil is another possibility to use
and improve it. <br></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>
When finally implemented - llvm-dwarfutil should solve the above
three issues and there <br>
would probably be more reasons to include DWARFLinker into lld.</div></blockquote><div><br></div><div>Is it the case that if the code is built to support the "read an executable, output a new better executable" use-case, it will actually be what's needed for the "output an optimized executable while linking object files" use-case? I worry that those could have enough different requirements that you really need to be developing the linker-integrated version from the very beginning in order to get a good result, rather than trying to shoehorn it in as an afterthought.</div></div></div></blockquote><div><br>Fair concern. I think there's probably a good chance of a lot of overlap in functionality/benefits - but, yes, likely some unspecified amount that would be context-dependent/different between lld/dwz/dwp/dsymutil use cases that are all slightly different.<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>Even if we would have the best solution - it is still useful to have
a tool like llvm-dwarfutil</div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>
for cases when it is necessary to process already created binaries. </div></blockquote><div><br></div><div>Sure -- I just think that should be considered as a secondary use-case, and not the primary goal.</div><div><br></div></div></div>
</blockquote></div></div>