<div dir="ltr"><div>I'd be happy with the body of llvm-objcopy being moved into a library regardless of the outcome of llvm-dwarfutil. The big advantage from that would be that we can write gtest unit tests for the library components rather than having to coerce an input to trigger the exact behaviour we want with lit. One of last year's GSOC projects, worked on by Alex Brachet-Mialot (CC'ed),
was to create a more general-purpose object editing library that objcopy would use. See also <a href="https://bugs.llvm.org/show_bug.cgi?id=41044">https://bugs.llvm.org/show_bug.cgi?id=41044</a>.</div><div><br></div><div>James<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 14 Sep 2020 at 12:17, Alexey via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Debuginfo folks,</p>
<p>What is your opinion on this proposal? <br>
</p>
<p>Do we need to work on better DWARFLinker library for now? or Can
we start to integrate llvm-dwarfutil as a series of small patches?<br>
</p>
<p>If it is OK to start integrating of llvm-dwarfutil, Is it OK to
move llvm-objcopy implementation into separate library ObjCopy ? <br>
</p>
<p>Thank you, Alexey.<br>
</p>
<div>On 01.09.2020 20:18, James Y Knight
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr"><br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Aug 31, 2020 at
11:06 AM Alexey <<a href="mailto:avl.lapshin@gmail.com" target="_blank">avl.lapshin@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> Hi James,<br>
<br>
Thank you for the comments. <br>
<br>
>I think we're not terribly far from that ideal, now,
for ELF. Maybe only these three things need to be done? --<br>
> 1. Teach lld how to emit a separated debuginfo
output file directly, without requiring an objcopy step.<br>
> 2. Integrate DWARFLinker into lld.<br>
> 3. Create a new tool which takes the separated
debuginfo and DWO/DWP files and uses DWARFLinker library <br>
> to create a new (dwarf-linked) separated-debug file,
that doesn't depend on DWO/DWP files.<br>
<br>
The three goals which you`ve described are our far goals.
<br>
Indeed, the best solution would be to create valid
optimized debug info without additional <br>
stages and additional modifications of resulting binaries.
<br>
<br>
There was an attempt to use DWARFLinker from the lld - <a href="https://reviews.llvm.org/D74169" target="_blank">https://reviews.llvm.org/D74169</a><br>
It did not receive enough support to be integrated yet.
There are fair reasons for that:<br>
<br>
1. Execution time. The time required by DWARFLinker for
processing clang binary is 8x bigger<br>
than the usual linking time. Linking clang binary with
DWARFLinker takes 72 sec, <br>
linking with the only lld takes 9 sec.<br>
<br>
2. "Removing obsolete debug info" could not be switched
off. Thus, lld could not use DWARFLinker for<br>
other tasks(like generation of index tables - .gdb_index,
.debug_names) without significant performance <br>
degradation.<br>
<br>
3. DWARFLinker does not support split dwarf at the moment.<br>
<br>
All these reasons are not blockers. And I believe
implementation from D74169 might be integrated and <br>
incrementally improved if there would be agreement on
that.</div>
</blockquote>
<div><br>
</div>
<div>Those do sound like absolutely critical issues for
deploying this for real -- whether as a separate tool or
integrated with lld. But possibly not critical enough to
prevent adding this behind an experimental flag, and working
on the code incrementally in-tree. However (without having
looked at the code in question), I wonder if the reported 8x
regression in link-time is even going to be salvageable just
by incremental optimizations, or if it might require a
complete re-architecting of the DwarfLinker code.</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>Using DWARFLinker from llvm-dwarfutil is another
possibility to use and improve it. <br>
</div>
</blockquote>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> When finally implemented - llvm-dwarfutil should solve
the above three issues and there <br>
would probably be more reasons to include DWARFLinker into
lld.</div>
</blockquote>
<div><br>
</div>
<div>Is it the case that if the code is built to support the
"read an executable, output a new better executable"
use-case, it will actually be what's needed for the "output
an optimized executable while linking object files"
use-case? I worry that those could have enough different
requirements that you really need to be developing the
linker-integrated version from the very beginning in order
to get a good result, rather than trying to shoehorn it in
as an afterthought.</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>Even if we would have the best solution - it is still
useful to have a tool like llvm-dwarfutil</div>
</blockquote>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> for cases when it is necessary to process already
created binaries. </div>
</blockquote>
<div><br>
</div>
<div>Sure -- I just think that should be considered as a
secondary use-case, and not the primary goal.</div>
<div><br>
</div>
</div>
</div>
</blockquote>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>