[lld] [lld][ELF] Introduce an option to keep data section prefix. (PR #148985)

[Mingming Liu via llvm-commits]

mingmingl-llvm wrote:

> On the linker side, the option should be generic, treating .hot and .unlikely uniformly. In LLVM CodeGen, a cl::opt option can be used to manage .hot and .unlikely settings.

We want to support two use cases:
1. with `-keep-data-section-prefix=true`, a section has three variants (hot, unlikely, original) in the executable
2. with `-keep-data-section-prefix=true -suppress-hot-data-section-prefix=true`, a section has two variants (unlikely, original) in the executable. The goal is to avoid the paranoid about the under-studied penalty of separating hot and lukewarm data. 

I agree that `-keep-data-section-prefix` should treat .hot and .unlikely uniformly. On the other hand, what do you think of adding another lld option like `-suppress-hot-data-section-prefix` as an alternative to the CodeGen option? I prototyped the codegen approach in https://github.com/llvm/llvm-project/pull/158172, yet I begin to realize the upsides of doing it in the linker and would like to shared them below and get some feedback, thanks in advance for the patience and time on this!

1. The intermediate object files can preserve three variants (hot, unlikely, original), making it possible to analyze, monitor and (if needed) debug the intermediate partition states.
2. It gives toolchain users a simpler mental model of reasoning about input -> output section mapping (all inside linker) using 'InputSection' as the abstractive. 
4. Generally, ease of maintenance and cross-platform (i.e., ELF/Mach-O/COFF) extension, less error-prone (to have uncovered .hot in the executable as a surprise).

To elaborate on 2 and 3 above, CodeGen is more prone to proliferate for the purpose of materializing various forms in-memory data representation into object files, especially support for new/customized data are added. Taking https://github.com/llvm/llvm-project/pull/158172  as an example, `getELFSectionNameForGlobal` line 660 - line 696 in llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp are modified to cover all known places where `.hot` data might be emitted. Generally it's not straightforward to decipher that line 679 - line 680 emits IR function attributes (e.g., `; Function Attrs: hot inlinehint mustprogress`) -- it's learnt with some logging/debugging. 

https://github.com/llvm/llvm-project/pull/148985