[llvm-dev] Fragmented DWARF

[James Henderson via llvm-dev]

(Resending with history trimmed to avoid it getting stuck in moderator
queue).

Hi Alexey,

Just an update - I identified the cause of the "Generated debug info is
broken" error message when I tried to build things locally: the
`outStreamer` instance is initialised with the host Triple, instead of
whatever the target's triple is. For example, I build and run LLD on
Windows, which means that a Windows triple will be generated, and
consequently a COFF-emitting streamer will be created, rather than the
ELF-emitting one I'd expect were the triple information to somehow be
derived from the linker flavor/input objects etc. Hard-coding in my target
triple resolved the issue (although I still got the other warnings
mentioned from my game link).

I measured the performance figures using LLD patched as described, and
using the same methodology as my earlier results, and got the following:

Link-time speed (s):
+-----------------------------+---------------+
| Package variant             | GC 1 (normal) |
+-----------------------------+---------------+
| Game (DWARF linker)         |  53.6         |
| Game (DWARF linker, no ODR) |  63.6         |
| Clang (DWARF linker)        | 200.6         |
+-----------------------------+---------------+

Output size - Game package (MB):
+-----------------------------+------+
| Category                    | GC 1 |
+-----------------------------+------+
| DWARFLinker (total)         |  696 |
| DWARFLinker (DWARF*)        |  429 |
| DWARFLinker (other)         |  267 |
| DWARFLinker no ODR (total)  |  753 |
| DWARFLinker no ODR (DWARF*) |  485 |
| DWARFLinker no ODR (other)  |  268 |
+-----------------------------+------+

Output size - Clang (MB):
+-----------------------------+------+
| Category                    | GC 1 |
+-----------------------------+------+
| DWARFLinker (total)         | 1294 |
| DWARFLinker (DWARF*)        |  743 |
| DWARFLinker (other)         |  551 |
| DWARFLinker no ODR (total)  | 1294 |
| DWARFLinker no ODR (DWARF*) |  743 |
| DWARFLinker no ODR (other)  |  551 |
+-----------------------------+------+

*DWARF = just .debug_info, .debug_line, .debug_loc, .debug_aranges,
.debug_ranges.

Peak Working Set Memory usage (GB):
+-----------------------------+------+
| Package variant             | GC 1 |
+-----------------------------+------+
| Game (DWARFLinker)          |  5.7 |
| Game (DWARFLinker, no ODR)  |  5.8 |
| Clang (DWARFLinker)         | 22.4 |
| Clang (DWARFLinker, no ODR) | 22.5 |
+-----------------------------+------+

My opinion is that the time costs of the DWARF Linker approach are not
really practical except on build servers, in the current state of affairs
for larger packages: clang takes 8.8x as long as the fragmented approach
and 11.2x as long as the plain approach (without the no ODR option). The
size saving is certainly good, with my version of clang 51% of the total
output size for the DWARF linker approach versus the plain approach and 55%
of the fragmented approach (though it is likely that further size savings
might be possible for the latter). The game produced reasonable size
savings too: 62% and 74%, but I'd be surprised if these gains would be
enough for people to want to use the approach in day-to-day situations,
which presumably is the main use-case for smaller DWARF, due to improved
debugger load times.

Interesting to note is that the GCC 7.5 build of clang I've used these
figures with produced no difference in size results between the two
variants, unlike other packages. Consequently, a significant amount of time
is saved for no penalty.

I'll be interested to see what the time results of the DWARF linker are
once further improvements to it have been made.

Thanks,

James

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201105/cd8c557d/attachment.html>