Support dead-stripping in ELF objects

[Robinson, Paul]

From: Shankar Easwaran [mailto:shankare at codeaurora.org]
> On 4/2/2013 8:23 PM, Michael Spencer wrote:
> > On Thu, Mar 21, 2013 at 11:13 AM, Robinson, Paul
> > <Paul.Robinson at am.sony.com>wrote:
> >
> >> A while back I implemented some private changes to how relocations
> >> were handled in the ELF object writer, so that our linker could
> >> support dead-stripping.  Apparently MachO has a similar feature, so I
> >> leveraged some of what was done for that target, including reusing
> >> the
> >> assembler flag/directive.
> >>
> >> I'm told that there may be broader interest in supporting
> >> dead-stripping in ELF objects, so here's the patch.
> >>
> >> There is currently no in-tree ELF-using target that invokes this
> >> stuff; I'm putting it out there so anyone who's interested can take
> >> advantage of it in their own target/toolchain.
> >>
> >> Please review, and commit for me if it's okay.
> >>
> >> Thanks,
> >> --paulr
> >>
> > This is a good idea, but I think we need more for an effective and
> > portable
> > implementation.
> >
> > We need a detailed specification of what guarantees the compiler is
> > making
> > for the linker. 

Michael,
Understanding what guarantees the compiler is making is certainly good.
The goal of my patch is:
* Any reference to a function uses a relocation whose symbol is that
  function's symbol (not relaxed to .text+N, and in the case of a call
  within the same .text section, not relaxed to a fixed offset).
* Any reference to a statically allocated data item uses a relocation
  whose symbol is that variable's symbol (not relaxed to .data+N or
  whatever section it is).
* In-memory constants (.rodata*) count as data items and are treated the
  same way as a variable.

The use of function symbol relocations implicitly divides the .text
section into atoms identifiable by symbol name.  Thus individual functions
can be dead-stripped if there are no references to that function's symbol.
Data items and in-memory constants whose only references are from
dead-stripped functions can also be dead-stripped.

> > I've thought about this in terms of both deadstripping
> > and
> > how to make atomization of ELF files faster and have the following:
> >
> > Section flags:
> > * SHF_VALID_SIZE

A better flag name might be SHF_ATOMIZED (symbols can certainly all have
valid sizes without defining atoms for you).

> >    * There shall be a symbol with st_value of 0.
> Is this to deal with weak symbols ?

Shankar, no, it's part of the guarantee the entire section is covered 
by symbols that define subsections.

> >    * For each symbol, st_value + st_size shall be the st_value of
> > another
> > symbol, or sh_size (the end of the section).
> This wouldnot handle the case for hand coded assembly files, where the
> user forgets about specifying the size of the function
> (Not  a requirement though). This also doesnot handle the case when
> symbols in the section are aligned differently.

Shankar, I think if a hand-coded assembly file uses .subsections_via_symbols
then it is reasonable to require the coder to set things up correctly.  It
would be nice if the toolchain could notice when that wasn't true, but
at some point the toolchain can't reasonably detect all possible mistakes.

> >    * For each symbol, the range [st_value, st_value + st_size) shall
> > be
> > non-overlapping.
> What if the symbol is 0 sized ? the next symbol and the current symbol
> will be overlapping right ?

Shankar, I would define it as a 0-sized atom that does not overlap.
But I don't think the compiler can produce these.

> >    * If a relocation in this section has a target symbol which is in a
> > section with this flag set, r_addend must be less than st_size.

Michael, I think this bullet wants to be "If any relocation has a target
symbol which is defined in a section with this flag set, r_addend must be
less than st_size."  It doesn't matter which section is making the reference.

> >
> >    This is separate from SHF_SUBSECTION_VIA_SYMBOLS because it makes
> > atomization faster even in cases where symbol data cannot be moved.
> > This is
> > important as it allows the assembler to preform more relaxation to get
> > smaller output while still allowing for this optimization.

Michael, the net effect I see is that the compiler has defined symbols that
subdivide the section, and promises that there are no relocations that use
a symbol for one subsection but the target is in a different subsection.

I believe my patch does this for .text, .data, and .rodata* sections,
and so could set the section flag on those sections.  On the other hand,
my patch is mainly driven by symbol type, not the section-of-definition;
so the atomization occurs because that's just how the compiler works, not
because the assembler is paying attention to all the symbols for a given
section and causing them to define subsections.

I'm not clear on what value you gain from atoms that can't be independently
moved, though.

> >
> > * SHF_SUBSECTION_VIA_SYMBOLS - SHF_VALID_SIZE must also be set.
> >    * The range [st_value, st_value + st_size) of each symbol in the
> > section
> > may be moved to a different relative location.
> Isnt this same as the functionality that you get by using
> -ffunction-sections/-fdata-sections ?

Shankar, thank you, I was not aware of these options...  they implement
functionality similar to the actual subdividing of the sections, we'd 
still need a flag somewhere that said "things are set up the way you want."
Also my toolchain (at least) wants FUNC/OBJECT symbols rather than section
symbols, so I'll still need my patch.

> >
> > * SHF_DEADSTRIP - This section may be removed if it is dead. If
> > SHF_SUBSECTION_VIA_SYMBOLS is also set, symbols and their data may be
> > removed if dead.
> When will this be used ?
> 
> I think this is duplicate of what -ffunction-sections/-fdata-sections
> will provide. You can dead strip a section
> and the symbols contained in it, since the compiler has created a
> seperate section for that.
> >
> >    This is separate because the assembler may have preformed
> > relaxation
> > within a section, but you would still like the entire section to be
> > removed
> > if it is not referenced. This also provides a way to limit the effect
> > of
> > --gc-sections without hard coding the names of sections not to remove
> > (such
> > as .init and .fini).

Michael, again I'm looking for the use case where we genuinely want to
permit the possibilities you've identified:
* The object is atomized for you, but the atoms have to stick together in 
  their sections. [SHF_VALID_SIZE alone] (or SHF_ATOMIZED)
* The atoms can be shuffled, but not can be removed
  [SHF_SUBSECTIONS_VIA_SYMBOLS alone] [well, implies +SHF_ATOMIZED]
* Section is not atomized but can be removed as a whole
  [SHF_DEADSTRIP alone]
* Atoms can be selectively removed; this is the one with the identified
  use case...
  [SHF_SUBSECTIONS_VIA_SYMBOLS+SHF_DEADSTRIP]

Saying "the assembler could do this or that or the other thing" isn't
really all that convincing.  WHY would the assembler do all these weird
things?  Identifying dead-strippable atoms, sure, because then the linker 
can actually do something positive to reduce the size of the final image.
But identifying atoms that cannot be stripped? or even reordered? How is
that useful?  I'm willing to be persuaded but there needs to be some
actual utility to all these cases before we burn so many bits.

If the distinctions aren't useful, then one section flag that means
SHF_SUBSECTIONS_VIA_SYMBOLS+SHF_ATOMIZED+SHF_DEADSTRIP is enough.

> >
> > Symbol flags (which we'll have to find a place to embed, probably
> > st_other
> > which has 6 free bits):
> > * STF_NO_DEADSTRIP - Never remove this symbol or symbols it references
> > even
> > if it is in a SHF_SUBSECTION_VIA_SYMBOLS or SHF_DEADSTRIP section

Michael, you propose that we could have symbols that are never referenced,
that the linker doesn't know about a-priori, that aren't special in some
way such as the image entrypoint symbols, and that the compiler will
identify as one that it does not want to be eliminated?  Like what?
I'm happy to concede my ignorance but if you're just enumerating all the
possibilities then I think maybe a little YAGNI is appropriate.

> The linker will save space only if SHF_SUBSECTION_VIA_SYMBOLS is set,
> which otherwise doesnot get any benefit right ?
> 
> Shankar Easwaran
> 
> --
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
> hosted by the Linux Foundation
> 

--paulr