[llvm-dev] [RFC] - Deduplication of debug information in linkers (LLD).

[David Blaikie via llvm-dev]

On Wed, Dec 6, 2017 at 3:15 AM George Rimar <grimar at accesssoftek.com> wrote:

> >If you're interested in things you can do in the linker for this - you
> might consider something more aggressive: Fully DWARF aware deduplication.
> >
> >This could be done hopefully by reusing some of the code in the dsymutil
> implementation in LLVM.
> >
> >This would be much more effective (and without the possible
> context-sensitive tradeoffs) than using type units.
> >Though it'd possibly have a big tradeoff in link time and/or linker
> memory usage (I'm not sure how much dsymutil needs/uses of either).
>
> + Rui.
>
> I think LLD development direction vector currently is to avoid teaching
> linker about things it naturally should not be aware off.
>

*nod* That's been the historic ELF+DWARF approach, but both MacOS (with
dsyms+DWARF) and Windows (COFF+CodeView+PDB) don't do it that way, and
instead involve the linker to a degree.

Mostly I'm wondering if it'd be reasonable to (and if anyone would be
interested in doing it) do something more like the PDB support - fully
debug-aware linking.


> Like it should ideally work with sections as pieces and should not know
> about content. That is not always possible,
> for example we have to look inside .eh_frame to deuplicate FDEs, but that
> is probably what we would want to avoid in general.
>

Yeah, I can totally understand that & it's historically how it's been done,
so I'm not expecting a change there, just floating the idea.


>
> >It doesn't seem especially important to implement the DWARF5 types ->
> debug_info thing for this situation, the type units
> >as they are (in debug_types) offer the same size benefits here. But sure,
> if anyone wanted to implement it at some point, that'd be fine.
>
> But there is no .debug_types in DWARF5, so it is depricated approach as
> far I understand.
>

Sure - but it works/is supported/is implemented. If someone wants to
implement the newer thing, that's cool, but I don't have any personal
motivation to do so for example. (& honestly we've been throwing around
some ideas about how to further generalize the debug_info contributions to
reduce some of the overhead of isolating types - so maybe if we're lazy
enough, we might leapfrog this particular state and just implement that
future better thing)


>
>
> >I think Paul covered some of the reasons type units might not be a
> reasonable default.
> >
> >One additional reason is that if you use Split DWARF (another great way
> to massively reduce the amount of debug info going to the linker)
> >type units are mostly /just/ overhead in the .dwo files: since the debug
> info is not linked, there's no opportunity to remove the
> >duplication anyway (unless you're making a DWP - like a >dsym file)
>
> Yeah. Looks -gsplit-dwarf​ and -fdebug-types-section are harmfull
> together. Probably it worth to restrict using of them together or
> emit a warning (both clang and gcc silently allows the combination and
> output has size penalty you describing).
>

Nah, only if you're not producing a DWP at the end (
https://gcc.gnu.org/wiki/DebugFissionDWP ).

In short, I probably wouldn't change any of LLVM's defaults. But there are
certainly flags people can use to reduce their debug info size.

You mentioned starting with this because LLVM's defaults mean the DWARF is
too large to link with DWARF 32 bit? How does gold cope with this? I
haven't seen failures/error messages/etc from either gold or lld related to
this? (though I mostly use Split DWARF myself)


>
> But then does it make sence to emit multiple .debug_info sections with
> -gsplit-dwarf, so that objects will contain skeleton .debug_info and
> .debug_info sections with type units as described in DWARF5. So
> that linker will be able to do deduplication of
> types on a sections level as expected ?
>
> George.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171206/483c7afa/attachment.html>