[llvm-dev] [Proposal][Debuginfo] dsymutil-like tool for ELF.

[David Blaikie via llvm-dev]

On Tue, Sep 1, 2020 at 10:18 AM James Y Knight <jyknight at google.com> wrote:

>
>
> On Mon, Aug 31, 2020 at 11:06 AM Alexey <avl.lapshin at gmail.com> wrote:
>
>> Hi James,
>>
>> Thank you for the comments.
>>
>> >I think we're not terribly far from that ideal, now, for ELF. Maybe only
>> these three things need to be done? --
>> >  1. Teach lld how to emit a separated debuginfo output file directly,
>> without requiring an objcopy step.
>> >  2. Integrate DWARFLinker into lld.
>> >  3. Create a new tool which takes the separated debuginfo and DWO/DWP
>> files and uses DWARFLinker library
>> > to create a new (dwarf-linked) separated-debug file, that doesn't
>> depend on DWO/DWP files.
>>
>> The three goals which you`ve described are our far goals.
>> Indeed, the best solution would be to create valid optimized debug info
>> without additional
>> stages and additional modifications of resulting binaries.
>>
>> There was an attempt to use DWARFLinker from the lld -
>> https://reviews.llvm.org/D74169
>> It did not receive enough support to be integrated yet. There are fair
>> reasons for that:
>>
>> 1. Execution time. The time required by DWARFLinker for processing clang
>> binary is 8x bigger
>> than the usual linking time. Linking clang binary with DWARFLinker takes
>> 72 sec,
>> linking with the only lld takes 9 sec.
>>
>> 2. "Removing obsolete debug info" could not be switched off. Thus, lld
>> could not use DWARFLinker for
>> other tasks(like generation of index tables - .gdb_index, .debug_names)
>> without significant performance
>> degradation.
>>
>> 3. DWARFLinker does not support split dwarf at the moment.
>>
>> All these reasons are not blockers. And I believe implementation from
>> D74169 might be integrated and
>> incrementally improved if there would be agreement on that.
>>
>
> Those do sound like absolutely critical issues for deploying this for real
> -- whether as a separate tool or integrated with lld. But possibly not
> critical enough to prevent adding this behind an experimental flag, and
> working on the code incrementally in-tree. However (without having looked
> at the code in question),
>

Yep, that's my feeling too.


> I wonder if the reported 8x regression in link-time is even going to be
> salvageable just by incremental optimizations, or if it might require a
> complete re-architecting of the DwarfLinker code.
>

Jonas, who's looked at llvm-dsymutil performance for its own sake
(motivated to improve llvm-dsymutil runtime, etc) & has mentioned on
this/related threads that there might be minimal headroom to improve things
there - so, yes, if there are greater opportunities it may require a fairly
large/broad investment (though a second/third set of eyes on the current
code to see if there are some hidden opportunities isn't a bad thing).


> Using DWARFLinker from llvm-dwarfutil is another possibility to use and
>> improve it.
>>
> When finally implemented - llvm-dwarfutil should solve the above three
>> issues and there
>> would probably be more reasons to include DWARFLinker into lld.
>>
>
> Is it the case that if the code is built to support the "read an
> executable, output a new better executable" use-case, it will actually be
> what's needed for the "output an optimized executable while linking object
> files" use-case? I worry that those could have enough different
> requirements that you really need to be developing the linker-integrated
> version from the very beginning in order to get a good result, rather than
> trying to shoehorn it in as an afterthought.
>

Fair concern. I think there's probably a good chance of a lot of overlap in
functionality/benefits - but, yes, likely some unspecified amount that
would be context-dependent/different between lld/dwz/dwp/dsymutil use cases
that are all slightly different.


> Even if we would have the best solution - it is still useful to have a
>> tool like llvm-dwarfutil
>>
> for cases when it is necessary to process already created binaries.
>>
>
> Sure -- I just think that should be considered as a secondary use-case,
> and not the primary goal.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200901/9b10aa2a/attachment-0001.html>