<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><br>
</p>
<div class="moz-cite-prefix">25.09.2019 9:21, Rui Ueyama пишет:<br>
</div>
<blockquote type="cite"
cite="mid:CAJENXgt1rGLUTa4r8TjAMA+L1TA7ouj15jTXDNXY+ppB9Tevvg@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Alexay,
<div><br>
</div>
<div>Thank you for the detailed explanation. The other question
I have is, as discussed above, about dsymutil. You said that
dsymutil is not usable at link-time. What does that mean? If
we only have to emit an output file in the usual way and then
automatically invoke dsymutils on the file that the linker
just created, that's easy to do, and lld and dsymutil can live
in the same process so that you can keep the linker being not
depend on an external command.</div>
</div>
</blockquote>
<p><span style="color: rgb(0, 0, 0); font-family: Calibri, Arial,
Helvetica, sans-serif; font-size: 16px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: 400; letter-spacing: normal; orphans: 2;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; background-color: rgb(255, 255,
255); text-decoration-style: initial; text-decoration-color:
initial; display: inline !important; float: none;">Calling
dsymutil as a separate invocation would have significant
overhead. Because linker would create a not optimized binary,
later dsymutil would load it again, parse, write optimized info
out. Thus, there would be additional disk space usage and
performance loss. When I said that dsymutil is not usable at<br>
link-time, I meant a more tight integration case. It is not
possible to reuse the implementation of dsymutil inside linker
at the current moment. It is not possible to say(somewhere after
markLive<ELFT>()) :<br>
<br>
DwarfLinker.link(DebugMap) // DwarfLinker is a dsymutil class<br>
<br>
and have the benefit of already loaded object files, already
created DebugMap and the stripped debug info sections by
DwarfLinker.link(), which later would be written out by lld.<br>
<br>
Presented PoC implementation, in contrast, uses benefits of
already loaded object files, created liveness information,
reduced output binary. It allows achieving minimal disk usage
and maximal execution speed.</span><span style="color: rgb(0, 0,
0); font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 16px; font-style: normal; font-variant-ligatures:
normal; font-variant-caps: normal; font-weight: 400;
letter-spacing: normal; orphans: 2; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(255, 255, 255); text-decoration-style:
initial; text-decoration-color: initial; display: inline
!important; float: none;"></span></p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CAJENXgt1rGLUTa4r8TjAMA+L1TA7ouj15jTXDNXY+ppB9Tevvg@mail.gmail.com"><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Sep 25, 2019 at 7:05
AM Alexey Lapshin <<a href="mailto:a.v.lapshin@mail.ru"
moz-do-not-send="true">a.v.lapshin@mail.ru</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p><br>
</p>
<div>24.09.2019 8:26, Rui Ueyama пишет:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hi Alexey,</div>
<div><br>
</div>
<div>Thank you for sharing this proposal. Reducing the
size of debug info is generally a good thing, and I
believe you'd see more debug info size reduction in
Rust programs than in C++ programs, because I heard
that the Rust compiler driver passes a lot of object
files to the linker, expecting that the linker would
remove most of them, which leaves dead debug info.</div>
<div dir="ltr"><br>
</div>
</div>
</blockquote>
<p>Hi Rui, Thanks!<br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">On Thu, Sep 12, 2019 at 7:32 AM Alexey
Lapshin via llvm-dev <<a
href="mailto:llvm-dev@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-dev@lists.llvm.org</a>>
wrote:<br>
</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>Debuginfo and linker folks, we (AccessSoftek)
would like to suggest a proposal for removing
obsolete debug info. If you find it useful we will
be happy to work on improving it. Thank you for
any opinions and suggestions.<br>
<br>
Alexey.<br>
<br>
Currently when the linker does garbage
collection a lot of abandoned debug info is left
behind (see Appendix A for documentation). Besides
inflated debug info size, we ended up with
overlapping address ranges and no way to say valid
vs garbage ranges. We propose removing debug info
along with removing code. This would reduce debug
info size and make sure debug info accuracy.<br>
<br>
There are several approaches which could be used
to solve that problem:<br>
<br>
1. Require dwarf producers to generate fragmented
debug data according to DWARF5 specification:
"E.3.3 Single-function-per-DWARF-compilation-unit"
page 388. That approach assumes fragmenting the
whole debug info per function basis and glue
fragmented sections at the link time using section
groups.<br>
<br>
2. Use an additional tool, which would optimize
out unnecessary debug data, something similar to
dwz (dwarf compressor tool), dsymutil (links the
DWARF debug information). This approach assumes
additional post-link binaries processing.<br>
<br>
3. Teach the linker to parse debug data and let
it remove unused debug data. <br>
<br>
In this proposal, we focus on approach #3. We show
that this approach is viable and discuss some
preliminary results, leaving particular
implementation out of the scope. We attach the
Proof of Concept (PoC) implementation(<a
href="https://reviews.llvm.org/D67469"
target="_blank" moz-do-not-send="true">https://reviews.llvm.org/D67469</a>)
for illustrative purposes. Please keep in mind
that it is not final, and there is room for
improvements (see Appendix B). However, the
achieved results look quite promising and
demonstrate up to 2 times size reduction and
performance overhead is 30% of linking time (which
is in the same ballpark as the already done
section compressing (see table 2 point F)).<br>
</div>
</blockquote>
<div><br>
</div>
<div>I believe #1 was added to DWARF5 to make
link-time debug info GC possible, so could you tell
me a little bit about why you chose to do #3? Is
this because you want to do this for DWARF4?</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div><br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<p><span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">No,
that proposal is not DWARF-4 specific. The proposal is
for DWARF-5 also. The solution added to DWARF-5("E.3.3
Single-function-per-DWARF-compilation-unit" page 388.)
is not a complete solution. This is a recommendation
which needs to have an additional specification.</span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">There
is -fdebug-types-section implementation which follows
that recommendation. Other cases(other than type units)
do not easily fit into this recommendation. There are
tables which have a common header. F.e. .debug_line,
.debug_rnglists, .debug_addr. It is not clear how these
tables could be separated between section groups.</span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">The
more important thing is the fragmentation itself.
Dividing debug tables into pieces would increase debug
info size.</span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">It
also would significantly complicate code working with
debug info. F.e.
include/llvm/DebugInfo/DWARF/DWARFObject.h has interface
for class DWARFObject. </span><span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">It
currently is not ready for the case when there could be
multiple tables. Patch introducing support for multiple
tables would be massive change affected many places in
llvm codebase. <br>
</span></p>
<p><span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Another
thing is that not only the llvm code base but all other
DWARF consumers should be changed to support fragmented
debug info. </span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Shortly,
if all debug tables would be fragmented then working
with debug info would be significantly complicated.</span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Thus
the reasons to select #3 are :</span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">1.
It could be done in a single place, not affecting other
parts of the llvm code base.</span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">2.
It does not require other DWARF consumers to implement
support for it.</span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">3.
Avoiding fragmentation would save space.</span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">4.
Processing of not fragmented debug info is faster.</span><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">5.
No need to adapt DWARF tables for fragmentation. They
could be handled with their current state.</span><br>
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<span
style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:16px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br>
</span></span></p>
<p>Alexey<br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div><br>
<br>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
<a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>