[llvm-dev] [LLD] Support DWARF64, debug_info "sorting"

[Fangrui Song via llvm-dev]

On 2020-11-12, Alexander Yermolovich wrote:
>Thanks for feedback.
>
>I agree with patch and numbers this will be a more concrete discussion, but I wanted to judge overall receptiveness to this approach and see maybe there was a better way.
>
>"Whilst the majority of objects will only have a single CU in them, there will be exceptions (LTO-generated objects, -r merged objects etc), so we do need to consider this approach."
>David can you elaborate under which conditions LTO-generated objects will have a mix of DWARF32/64 in same .debug_info? Looking at how dwarf64 was implemented same flag will be used for the entirety of the dwarf output, even if multiple CUs are included.
>
>I think if object does have a mix of CUs that are 32/64, linker can do a best effort ordering, and output a warning. My approach to this is from covering common cases while solving a problem with relocations overflow in large libraries/binaries.
>
>
>@Fangrui Song<mailto:maskray at google.com>
>That's a good point with relocations. Although is it always a guarantee a first one will be representative of entire relocation record?
>For debug_info even with DWARF32 there can be 64bit relocations.
>0000000000000c57  0000001800000001 R_X86_64_64            0000000000000000 .text._"some_mangeled_name" + 0

It may be weaker than "guaranteed": working in practice.

Let's look at sections that reference these large .debug_* sections (.debug_info, .debug_str, .debug_loclists, .debug_rnglists, ...):

* .debug_info: the first relocation references .debug_abbrev, good indicator
* .debug_names references .debug_info: the first relocation (CU offset) is a good indicator
* .debug_aranges references .debug_info: the first relocation (debug_info_offset) is a good indicator
* .debug_str_offsets references .debug_str: the first relocation (.debug_str offset) is a good indicator
* ...

So checking the first relocation is probably sufficient. Even if we miss
something, we can adjust the heuristic, or rather let the compiler generate an
artificial relocation (R_*_NONE), which will always work.

>On one hand since this is only applicable for when DWARF64 is used, special option would be the way to go. Although the user will need to be aware of yet another LLD option. Maybe an error when relocations overflow occur can be modified to display this option along with -fdebug-types-section

I forgot to mention another drawback with .debug_* parsing. In the
presence of compressed debugging information, currently we uncompress
.debug_* on demand. We usually do it when writing the content of the
output section, which means we can potentially discard the uncompressed
buffers after we have done processing with one output section and move
to the next. This trick can potentially save peak memory usage.

However, if we do .debug_* parsing (to decide ordering among DWARF32/DWARF64),
we either cache the result (lose the trick) or end up uncompressing twice.
Neither is good.



I am quite happy with the relocation approach under a linker option. I'd still
want to know generic-abi folks's thoughts, though. James may have prepared something
he wants to share with generic-abi:) Let's wait...