<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi James,</p>
On 05.11.2020 17:59, James Henderson wrote:<br>
<blockquote type="cite"
cite="mid:CABqSp3mjeCq-wvdHDA93dyB0zLKFhY3bFui0xhfqUoaS1Tzf2g@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="ltr">(Resending with history trimmed to avoid it
getting stuck in moderator queue).<br>
</div>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr"><br>
</div>
<div>
<div>Hi Alexey,</div>
<div><br>
</div>
<div>Just an update - I identified the cause of the
"Generated debug info is broken" error message when I
tried to build things locally: the `outStreamer` instance
is initialised with the host Triple, instead of whatever
the target's triple is. For example, I build and run LLD
on Windows, which means that a Windows triple will be
generated, and consequently a COFF-emitting streamer will
be created, rather than the ELF-emitting one I'd expect
were the triple information to somehow be derived from the
linker flavor/input objects etc. Hard-coding in my target
triple resolved the issue (although I still got the other
warnings mentioned from my game link).</div>
</div>
</div>
</div>
</blockquote>
<p> Thank you for the details. Actually, I did not test this on
Windows. But I would do and update the patch.</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CABqSp3mjeCq-wvdHDA93dyB0zLKFhY3bFui0xhfqUoaS1Tzf2g@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div>
<div><br>
</div>
<div>I measured the performance figures using LLD patched as
described, and using the same methodology as my earlier
results, and got the following:</div>
<br>
<div>
<div>Link-time speed (s):</div>
<div><span style="font-family:monospace">+-----------------------------+---------------+</span><br>
</div>
<div><font face="monospace">| Package variant
| GC 1 (normal) |</font></div>
<div><font face="monospace">+-----------------------------+---------------+<br>
</font></div>
<div><font face="monospace">| Game (DWARF linker)
| 53.6 |
</font>
<div><font face="monospace">| Game (DWARF linker, no
ODR) | 63.6 |</font></div>
</div>
<div><font face="monospace">| Clang (DWARF linker)
| 200.6 |</font></div>
<div><font face="monospace">+-----------------------------+---------------+</font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace">
</font>
<div><font face="monospace"><font
face="arial,sans-serif">Output size - Game package
(MB):</font></font></div>
<div><font face="monospace"><font
face="arial,sans-serif"><span
style="font-family:monospace">+-----------------------------+------+</span><br>
</font></font></div>
<div><span style="font-family:monospace">| Category
| GC 1 |<br>
</span></div>
<div><span style="font-family:monospace">+-----------------------------+------+<br>
</span></div>
<div><span style="font-family:monospace">| DWARFLinker
(total) | 696 |<br>
</span></div>
<div><span style="font-family:monospace">| DWARFLinker
(DWARF*) | 429 |<br>
</span></div>
<div><span style="font-family:monospace">| DWARFLinker
(other) | 267 |
</span>
<div><span style="font-family:monospace">| DWARFLinker
no ODR (total) | 753 |<br>
</span></div>
<div><span style="font-family:monospace">| DWARFLinker
no ODR (DWARF*) | 485 |<br>
</span></div>
<div><span style="font-family:monospace">| DWARFLinker
no ODR (other) | 268 |</span></div>
</div>
<div><font face="monospace">+-----------------------------+------+</font>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace"><font
face="arial,sans-serif">Output size - Clang
(MB):
</font></font>
<div><font face="monospace"><font
face="arial,sans-serif"><span
style="font-family:monospace">+-----------------------------+------+</span><br>
</font></font></div>
<div><span style="font-family:monospace">| Category
| GC 1 |<br>
</span></div>
<div><span style="font-family:monospace">+-----------------------------+------+</span></div>
<div><span style="font-family:monospace">|
DWARFLinker (total) | 1294 |<br>
</span></div>
<div><span style="font-family:monospace">|
DWARFLinker (DWARF*) | 743 |<br>
</span></div>
<div><span style="font-family:monospace">|
DWARFLinker (other) | 551 |
<span style="font-family:monospace">
</span></span>
<div><span style="font-family:monospace">|
DWARFLinker no ODR (total) | 1294 |<br>
</span></div>
<div><span style="font-family:monospace">|
DWARFLinker no ODR (DWARF*) | 743 |<br>
</span></div>
<div><span style="font-family:monospace">|
DWARFLinker no ODR (other) | 551 |
</span>
<div>
<div><span style="font-family:monospace"></span></div>
</div>
<div><font face="monospace">+-----------------------------+------+</font></div>
</div>
</div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace"><span
style="font-family:arial,sans-serif">*DWARF =
just .debug_info, .debug_line, .debug_loc,
.debug_aranges, .debug_ranges.</span></font></div>
<div><font face="monospace"><span
style="font-family:arial,sans-serif"></span></font><br>
<font face="monospace"><span
style="font-family:arial,sans-serif"></span>
</font>
<div><span style="font-family:monospace"><span
style="font-family:arial,sans-serif">Peak
Working Set Memory usage (GB):</span><br>
</span></div>
<div><span style="font-family:monospace">
</span>
<div><font face="monospace"><font
face="arial,sans-serif"><span
style="font-family:monospace">+-----------------------------+------+
</span></font></font></div>
<div><span style="font-family:monospace">
| Package variant | GC 1 |<br>
</span></div>
<div><span style="font-family:monospace">
+-----------------------------+------+<br>
</span></div>
<div><span style="font-family:monospace">
| Game (DWARFLinker) | 5.7 |
</span>
<div><span style="font-family:monospace">
| Game (DWARFLinker, no ODR) | 5.8 |</span></div>
</div>
<div><span style="font-family:monospace">
| Clang (DWARFLinker) | 22.4 |
</span>
<div>
<div><span style="font-family:monospace"></span></div>
</div>
<div><span style="font-family:monospace">
| Clang (DWARFLinker, no ODR) | 22.5 |</span></div>
</div>
<div><span style="font-family:monospace">
+-----------------------------+------+</span></div>
<div><span style="font-family:monospace"><br>
</span></div>
<div><span style="font-family:monospace"><span
style="font-family:arial,sans-serif">My
opinion is that the time costs of the
DWARF Linker approach are not really
practical except on build servers, in the
current state of affairs for larger
packages: clang takes 8.8x as long as the
fragmented approach and 11.2x as long as
the plain approach (without the no ODR
option). The size saving is certainly
good, with my version of clang 51% of the
total output size for the DWARF linker
approach versus the plain approach and 55%
of the fragmented approach (though it is
likely that further size savings might be
possible for the latter). The game
produced reasonable size savings too: 62%
and 74%, but I'd be surprised if these
gains would be enough for people to want
to use the approach in day-to-day
situations, which presumably is the main
use-case for smaller DWARF, due to
improved debugger load times.<br>
</span></span></div>
<div><span style="font-family:arial,sans-serif"><br>
</span></div>
<div><span style="font-family:monospace"><span
style="font-family:arial,sans-serif">Interesting
to note is that the GCC 7.5 build of clang
I've used these figures with produced no
difference in size results between the two
variants, unlike other packages.
Consequently, a significant amount of time
is saved for no penalty.</span><br>
</span></div>
<div><span style="font-family:arial,sans-serif"><br>
</span></div>
<div><span style="font-family:monospace"><span
style="font-family:arial,sans-serif">I'll
be interested to see what the time results
of the DWARF linker are once further
improvements to it have been made.<br>
</span></span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<p>yep, current time costs of the DWARFLinker are too high. One of
the reasons is that lld handles sections in parallel, while
DWARFLinker handles data sequentially. Probably DWARFLinker
numbers could be improved if it would be possible to teach it to
handle data in parallel. Thank you for the comparison!<br>
</p>
<p>Speaking of "Fragmented DWARF" solution, how do you estimate
memory requirements to support fragmented object files ? In
comments for your Lightning Talk you have mentioned that it would
be necessary to "<span id="6345162" class="message left"
data-tip="8:46 PM" data-place="right">update DebugInfo library
to treat the fragmented sections as one continuous section</span>".
Do you think it would be cheap to implement?<br>
</p>
<p>Thank you, Alexey.<br>
</p>
<blockquote type="cite"
cite="mid:CABqSp3mjeCq-wvdHDA93dyB0zLKFhY3bFui0xhfqUoaS1Tzf2g@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div><span style="font-family:arial,sans-serif"><br>
</span></div>
<div><span style="font-family:arial,sans-serif">Thanks,<br>
</span></div>
<div><span style="font-family:arial,sans-serif"><br>
</span></div>
<div><span style="font-family:monospace"><span
style="font-family:arial,sans-serif">James</span><br>
</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<blockquote class="gmail_quote" style="margin:0px
0.8ex;border-left:1px solid
rgb(204,204,204);border-right:1px solid
rgb(204,204,204);padding-left:1ex;padding-right:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</body>
</html>