[PATCH] D60242: Add IR support, ELF section and user documentation for partitioning feature.

[ben via Phabricator via llvm-commits]

bd1976llvm added a comment.

In D60242#1456509 <https://reviews.llvm.org/D60242#1456509>, @pcc wrote:

> In D60242#1456022 <https://reviews.llvm.org/D60242#1456022>, @bd1976llvm wrote:
>
> > Hi Peter,
> >
> > I really like the concept. I had some nebulous concerns when I read the RFC; but, I didn't comment as I couldn't think of any alternatives to what you were proposing. What I am writing here is still not fully formed.. so feel free to ignore if it is unhelpful.
> >
> > My main concern is that this is supporting a rare usecase and I don't really like the idea of adding complexity to the core tools for niche cases.
> >
> > I feel like it would be better if this could be implemented via post-processing in some way; outside of the core tools.
> >
> > Some aspect of this, like the linker being able to report all of the functions reachable from  a given set of entry points, are of general utility though.
> >
> > I wonder if there is any scope for implementing something different along the lines of:
> >
> > 1. Do an initial link.
> > 2. Analyse the resulting elf and report all of the functions in each partition.
> > 3. Do another link but this time provide an ordering file so that all of the functions for each partition are placed contiguously.
> > 4. Extract the functions for each partition, apart form the first, into a partition file which also contains metadata specifying the address range.
> > 5. Write zeros over all of the functions apart from those in the main partition and then compress the executable (saving the file space).
> > 6. A loader extension patches in the missing functions for each partition when needed.
>
>
> Hi Ben, thanks for the feedback. I shared a similar concern around complexity, but I don't see any reasonable alternatives to implementing this directly in the linker.
>
> Your idea of using orderfiles is interesting, but I see at least two showstopper problems with it. The first is that it is incompatible with range extension thunks. Imagine that you have one main partition and two loadable partitions. An orderfile is used to sort the main partition's sections before partition 1's sections before partition 2's sections. We're going to split partition 1 and partition 2 into separate files, but the linker doesn't know that, so it may place a range extension thunk in the middle of partition 1's sections and have partition 2 call it. Now at runtime we will crash if partition 2 is loaded without partition 1 and it tries to call one of the range extension thunks in partition 1. We might be able to work around this issue somehow by telling the linker where the partition boundaries are, but at that point since the linker needs to know how you're going to split the file up anyway it might as well actually split it up for you.
>
> The second is that replacing parts of the file with zeros and compressing it isn't always useful from a size perspective; in many cases it's the uncompressed size that matters because that's what takes up storage space on the device. One might consider uncompressing the code at program startup but that leads to other problems: it slows down startup and prevents the kernel from paging out the pages containing the uncompressed code.
>
> I also see other problems that aren't showstoppers but are still significant:
>
> - We're leaving some of the binary size gains on the table because we can't move exception-handling sections, dynamically relocatable sections (e.g. vtables) or symbol tables.
> - Since we aren't splitting the symbol table the solution is more error prone since there's nothing stopping someone from dlsym'ing a symbol in an unloaded partition.
> - This requires linking multiple times, which will slow down linking and hurt developer productivity, but the problem is made worse when (full) LTO is being used since we'll need to do LTO multiple times.


Thanks for considering this carefully. IMO most of your concerns could be accepted as reasonable limitations to keep the feature out of the linker. We could probably deal with the points that you raised with the compression scheme somehow (e.g. some horrible scheme that adjusts the existing program headers and adds an additional bss one, similar to your original proposal but done by post processing). However, I think your point about thunks is a hard showstopper. We would need a much more expressive means of controlling the linkers placement of things (than the current ordering files, linker scripts, --section-start etc..) in order to create a layout that would be amenable to post processing; and, as you hinted at, this would be just as much work (or more) than just getting the linker to do the splitting up.

Given the above, I am happy with the implementation.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D60242/new/

https://reviews.llvm.org/D60242