[PATCH] D60242: Add IR support, ELF section and user documentation for partitioning feature.

[Peter Collingbourne via Phabricator via llvm-commits]

pcc added a comment.

In D60242#1456022 <https://reviews.llvm.org/D60242#1456022>, @bd1976llvm wrote:

> Hi Peter,
>
> I really like the concept. I had some nebulous concerns when I read the RFC; but, I didn't comment as I couldn't think of any alternatives to what you were proposing. What I am writing here is still not fully formed.. so feel free to ignore if it is unhelpful.
>
> My main concern is that this is supporting a rare usecase and I don't really like the idea of adding complexity to the core tools for niche cases.
>
> I feel like it would be better if this could be implemented via post-processing in some way; outside of the core tools.
>
> Some aspect of this, like the linker being able to report all of the functions reachable from  a given set of entry points, are of general utility though.
>
> I wonder if there is any scope for implementing something different along the lines of:
>
> 1. Do an initial link.
> 2. Analyse the resulting elf and report all of the functions in each partition.
> 3. Do another link but this time provide an ordering file so that all of the functions for each partition are placed contiguously.
> 4. Extract the functions for each partition, apart form the first, into a partition file which also contains metadata specifying the address range.
> 5. Write zeros over all of the functions apart from those in the main partition and then compress the executable (saving the file space).
> 6. A loader extension patches in the missing functions for each partition when needed.


Hi Ben, thanks for the feedback. I shared a similar concern around complexity, but I don't see any reasonable alternatives to implementing this directly in the linker.

Your idea of using orderfiles is interesting, but I see at least two showstopper problems with it. The first is that it is incompatible with range extension thunks. Imagine that you have one main partition and two loadable partitions. An orderfile is used to sort the main partition's sections before partition 1's sections before partition 2's sections. We're going to split partition 1 and partition 2 into separate files, but the linker doesn't know that, so it may place a range extension thunk in the middle of partition 1's sections and have partition 2 call it. Now at runtime we will crash if partition 2 is loaded without partition 1 and it tries to call one of the range extension thunks in partition 1. We might be able to work around this issue somehow by telling the linker where the partition boundaries are, but at that point since the linker needs to know how you're going to split the file up anyway it might as well actually split it up for you.

The second is that replacing parts of the file with zeros and compressing it isn't always useful from a size perspective; in many cases it's the uncompressed size that matters because that's what takes up storage space on the device. One might consider uncompressing the code at program startup but that leads to other problems: it slows down startup and prevents the kernel from paging out the pages containing the uncompressed code.

I also see other problems that aren't showstoppers but are still significant:

- We're leaving some of the binary size gains on the table because we can't move exception-handling sections, dynamically relocatable sections (e.g. vtables) or symbol tables.
- Since we aren't splitting the symbol table the solution is more error prone since there's nothing stopping someone from dlsym'ing a symbol in an unloaded partition.
- This requires linking multiple times, which will slow down linking and hurt developer productivity, but the problem is made worse when (full) LTO is being used since we'll need to do LTO multiple times.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D60242/new/

https://reviews.llvm.org/D60242