[llvm-dev] [RFC] Debug sections for hot-patching LLD's ELF output

Thu Sep 16 07:16:31 PDT 2021

Thanks Peter,

On Thu, Sep 16, 2021 at 9:30 AM Peter Smith <Peter.Smith at arm.com> wrote:

> Thanks for sending this out.
>
>
>
> My initial reaction is that this would be most useful for post linking
> tools. For human readable output only I expect that we’d be comfortable
> with existing map file output and a disassembly.
>

Emitting a linkmap section is *much* more efficient than generating a -Map
file. Testing with a chromium package on my windows development box, I got
the following results using a build of main with
https://reviews.llvm.org/D109804 applied: Base link = 3.524 s, With -Map =
8.441 s, With --debug-sections (for the linkmap section) = 3.910 s. Also,
with the linkmap section the information is built into the ELF so you don't
have the problem of tracking which -Map file goes with which ELF. Given
those advantages, for some use-cases the linkmap is a superior solution -
even when only humans will want to view the information.

> I have a small concern of upstream maintainability without the binary
> patching tools themselves. For example it may be that all we have is the
> llvm-readobj/llvm-objdump to textually dump the output. It is possible that
> we could make modifications with corresponding changes to the text dumps
> that could break assumptions the binary patching tools are making. I think
> this is likely to be rare, but I couldn’t rule it out.
>

>
> While I wouldn’t object as I think the extra debug output is not likely to
> need a lot of maintenance I think it would be good to get someone actively
> interested in binary patching or some other post-link tool to comment.
>

Unfortunately, hotpatching seems to be more of a propriety technology
rather than opensource (although it would be great to be wrong and I'm
hoping that someone may reply and correct me).

Sony and partners are interested in these sections for our downstream
toolchain and we will commit to monitoring them and ensuring the format
doesn't change without agreement. We could also add upstream tests that
check a hex dump of the section contents for these debugging sections and
which have strongly worded comments in the test noting that the format
should not change without agreement from stakeholders (we actually
successfully use a similar strategy in our downstream tests to flag up
upstream changes for review).

I envision that these linker debug sections will have a version field (as
DWARF sections do) which we can use to manage changes to the format of
these sections.

> Peter
>
>
>
> *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> *On Behalf Of *bd1976
> llvm via llvm-dev
> *Sent:* 15 September 2021 03:52
> *To:* llvm-dev <llvm-dev at lists.llvm.org>
> *Subject:* [llvm-dev] [RFC] Debug sections for hot-patching LLD's ELF
> output
>
>
>
> Hi All,
>
> Sony maintains a downstream patchset to optionally emit additional
> informational sections to the ELF output file created by LLD. These
> sections describe LLD's output and the transformations applied during
> Linking. These additional sections are used with the static symbol
> table (.symtab) to facilitate the operation of hot-patching tools.
>
> Our preferences are that:
>
> - The information required for hot-patching is stored in the ELF
>   output file as ELF sections, as opposed to being emitted into
>   auxiliary files. Otherwise, customers have to adjust their processes
>   to keep the ELF output file and auxiliary files together when
>   packing/moving the ELF output file and ensure they are correctly
>   matched.
>
> - These metadata sections are created by LLD, rather than derived via
>   a post-link procedure. Performance is important, as customers want
>   to be able to enable the emission of hot-patching metadata by
>   default, and having LLD directly emit the required sections is more
>   efficient and a simpler work-flow.
>
> The contents of these sections could be seen as debugging information
> for the linking process. Certainly, we would want to handle these
> sections with the same rules that apply to debugging sections when
> manipulating a linked ELF with binary utility tools. For that reason
> the sections are all named .debug_lld_* e.g. .debug_lld_linkmap.
>
> Currently, Sony would like to emit the following sections and we
> believe that they are generally useful:
>
> - A linkmap section that contains a subset of the information contained
>   in a linker -Map file. This section specifies the linked address for
>   each input section.
>
> - A section which specifies the list of wrapped symbols.
>
> - A section that describes the GOT. This provides:
> -- A category for each entry, examples: GOT entry, PLTGOT entry, TLS GD
>    entry, LD TLS tls_index structure entry etc..
> -- A slot index at which the entry starts.
> -- A size for the entry, as GOT entries may take more than one GOT
>    slot (e.g. a TLS GD entry takes two slots).
> -- An optional static symbol index to which the GOT entry is associated
>    (some entries e.g. the LD TLS tls_index structure are not associated
>    with a particular symbol).
>
> - A section describing the PLT. This section needs to be somewhat
>   flexible to deal with the many different PLT's that exist on ELF
>   toolchains. However, for a fixed size entry PLT description the section
>   will supply:
> -- Which range of bytes comprises the PLT header.
> -- The size of a PLT entry.
> -- For each PLT entry, the GOT slot index of the associated GOT entry.
>    Combined with the information on GOT entries from the GOT description
>    section this allows for the association of a PLT entry with a symbol.
>
> Similar to DWARF sections these are non-alloc sections. They are encoded
> as sequences of ULEB128 values. As these are debugging sections, not core
> ELF sections, a compact representation is justifiable, even if the encoding
> is more complex.
>
> In order to anchor this discussion I have created
> https://reviews.llvm.org/D109804
> which contains a prototype implementation of the linkmap section referenced
> above.
>
> I would like to ascertain whether the LLVM community would be
> supportive of adding the ability to generate such sections to LLD?
>
> Thanks.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210916/cf892222/attachment.html>