[patch][rfc] Implicit sections

[Shankar Easwaran]

On 2/11/2015 10:23 PM, Rafael Espíndola wrote:
> The linker in the ps4 sdk is able to split sections into smaller
> chunks for GC. It does so using the symbol size that is recorded in
> the ELF files (unlike MachO that uses the next symbol to find the
> boundary).
Isnt this same as the Atom model with lld, lld uses symbol size to 
construct atoms.

While this improves GC, what about :-

a) linker script rules.
b) how is COMDAT handled
c) How are section relative relocations translated ? I believe it uses a 
section symbol + offset, how would this be taken care.

> As part of the work of upstreaming the llvm changes, I have been
> looking at a way to view and implement this as a general and sound
> optimization.
>
> In order to be able to split sections, the linker needs extra
> relocations. In fact, it needs the same relocations as would be needed
> if -ffunction-sections and -fdata-sections were used.
Why dont we use -ffunction-sections and -fdata-sections by default.

What is the alignment of each individual section that has been split ?

> This led to the insight that a way to view this is just as a more
> compact .o representation of multiple sections. The codegen and
> assembler work as if -function-sections/-fdata-sections were used, but
> we can skip outputting the extra sections because we know that the
> linker can recreate them.
>
> The wip patch of the first step is attached. What it does is add
> support for multiple sections with the same name. It does so by adding
> a "unique" marker (bike sheds on the name welcome). This just means
> that every .section with the unique marker in the .s creates a new
> section.
>
> With the support in place, the patch then avoids creating multiple
> section names when using -ffunction-section and -fdata-section. The .o
> files just end up with many .text sections (for example).
>
> This should work with every ELF linker.
>
> The patch needs a bit more work (proper option instead of cl::opt,
> more compact encoding), but it can bootstrap and saves some space in
> the .o files. In a build dir after running "ninja && ninja check-all"
> the total size of the .o files goes from 237871087 bytes to 231292287.
I couldnt follow how does this save space ?

> The patch also offers some saving even when
> -ffunction-sections/-fdata-sections are not used by avoiding long
> names in comdats.
>
> The next step would then be to add an "implicit" flag in addition to
> the "unique" one. This would mean that when given
>
> .section .text,"ax", at progbits, implicit
> .section .text,"ax", at progbits, implicit
How does partial linking work ?
>
> MC would in the end output a single .text section, but would produce
> the extra relocation in order to make it possible to split them in the
> linker.
>
> The resulting file would still be a legal ELF, it would just require
> extra logic in the linker to be able to GC it. The tradeoff being the
> smaller .o files.
>
> I still don't have the numbers on what the saving from omitting the
> sections might be, but a as an upper bound, in a build with clang
> using this patch and no -ffunction-section/-fdata-sections the total
> .o size is 217519383 bytes.
>
> With this in then the default for PS4 would be changed to
> -ffunction-sections, -fdata-sections and implicit sections.
>
> The advantage of implementing it this way is that it makes it clear
> that it is really just a .o compactness optimization. No extra logic
> is required for deciding if a relocation is needed. The extra handling
> of "implicit" would be just in the ELF object writer.
Curious, is this planned in binutils too ?

Shankar Easwaran

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the Linux Foundation