[llvm] [AArch64, ELF] Allow implicit $d/$x at section beginning (PR #99718)

[Peter Smith via llvm-commits]

smithp35 wrote:

Some thoughts after thinking about this on the weekend and talking to some colleagues.

Apologies for the length of the response, a brief summary:
* I think this could be a useful option if projects build all their objects with it enabled. Would be good to know from potential users that they would enable this option in this review? So far I think it is only me that has commented.
* My main concern is binary distribution of objects with it enabled, and what future choices this might rule out.
* I'm still nervous about trailing mapping symbols, particularly with `-fsanitize=undefined` or `-fsanitize=kcfi`.
* Can we alleviate these concerns by marking object files that use this mapping symbol convention. That way we could fix them up later if needed. 
* Is there an alternative way to do link-time culling of mapping symbols that runs faster?

In more detail.

The pragmatist in me says that this is a useful (possibly llvm only [1]) option for people with large builds to enable an alternative mapping symbol definition than the ABI and accept the consequences of that as well as the benefits. The idealist in me says that if this propagates out into binary distributions across multiple projects then we may have incompatibilities with GNU and while there are no plans to add one, it may rule out something like a second instruction set (like Morello or Thumb [2]).
[1] the comments on the binutils mailing list don't look enthusiastic.
[2] We can't know which instruction set is implicit.

A colleague working on Bolt mentioned that there could be a case for more mapping symbols to help binary analysis, this implies that there could be more mapping symbol density options than just optimize and default. He also mentioned that recording the mapping symbol level choice in the ELF file (mechanism TBD) might be a way around the future compatibility problem, and could extend to the additional mapping symbol cases. Does recording the use of the alternative mapping symbol mechanism appeal at all?

I'm still nervous about adding trailing `$x`. I had thought that AArch64 executable sections with a leading `$d` would be very rare, but I had forgotten about `-fsanitize=kcfi` and `-fsanitize=undefined`. These all lead with a `$d`. This makes collisions between trailing `$x` and `$d` more likely than I'd first thought.

I also don't think we can rely on the dumb linker collation order putting the trailing `$x` before the leading `$d` of the following section. For example, assuming we have `lld first.o second.o` with the following linker script fragment: 
```
  .text : { second.o(.text) first.o(.text) }
```
If `.text` in first.o leads with a `$d`; like when `-fsanitize=kcfi` is used, and `.text` in second.o has a trailing `$x` then I think when we copy local symbols into the symbol table first.o will get copied first, which will be maintained by stable_partition so we'll get `$d` `$x` at the same address with the `$x` . This is all theoretical, I haven't tried testing it yet.

As an aside, I'm surprised the linker removal of redundant mapping symbols took so long. It looks like we already stable_partition the static symbol table in `void SymbolTableBaseSection::finalizeContents()`. Did you modify that to stable_sort rather than partition, or just stable_sort the local symbols after the partition?

Another possibility that I think would work in the majority of AArch64 cases without costing too much extra link time, which I think could be done before the stable_partition (to speed that up) is to see if each OutputSection always has the same mapping symbol type. For example if the `.text` section only has `$x` mapping symbols, which would be the most common case, then we can replace these with a single `$x` mapping symbol. If there any `$d` in there bail out. I think this could be done much more cheaply than 5% link time. 

https://github.com/llvm/llvm-project/pull/99718