[lld] [LLD] Implement --enable-non-contiguous-regions (PR #90007)

[Daniel Thornburgh via llvm-commits]

================
@@ -1364,6 +1432,107 @@ const Defined *LinkerScript::assignAddresses() {
   return getChangedSymbolAssignment(oldValues);
 }
 
+static bool isRegionOverflowed(MemoryRegion *mr) {
+  if (!mr)
+    return false;
+  return mr->curPos - mr->getOrigin() > mr->getLength();
+}
+
+// Spill input sections in reverse order of address assignment to (potentially)
+// bring memory regions out of overflow. The size savings of a spill can only be
+// estimated, since general linker script arithmetic may occur afterwards.
+// Under-estimates may cause unnecessary spills, but over-estimates can always
+// be corrected on the next pass.
+bool LinkerScript::spillSections() {
+  if (!config->enableNonContiguousRegions)
+    return false;
+
+  bool spilled = false;
+  for (SectionCommand *cmd : reverse(sectionCommands)) {
+    auto *od = dyn_cast<OutputDesc>(cmd);
+    if (!od)
+      continue;
+    OutputSection *osec = &od->osec;
+    if (!osec->size || !osec->memRegion)
+      continue;
+
+    DenseSet<InputSection *> spills;
+    for (SectionCommand *cmd : reverse(osec->commands)) {
+      if (!isRegionOverflowed(osec->memRegion) &&
+          !isRegionOverflowed(osec->lmaRegion))
+        break;
+
+      auto *is = dyn_cast<InputSectionDescription>(cmd);
+      if (!is)
+        continue;
+      for (InputSection *isec : reverse(is->sections)) {
+        // Potential spill locations cannot be spilled.
+        if (isa<SpillInputSection>(isec))
+          continue;
+
+        // Find the next spill location.
+        auto it = spillLists.find(isec);
+        if (it == spillLists.end())
+          continue;
+
+        spilled = true;
+        SpillList &list = it->second;
+
+        SpillInputSection *spill = list.head;
+        if (!spill->next)
+          spillLists.erase(isec);
+        else
+          list.head = spill->next;
+
+        spills.insert(isec);
+
+        // Replace the next spill location with the spilled section and adjust
+        // its properties to match the new location.
+        *llvm::find(spill->isd->sections, spill) = isec;
+        isec->parent = spill->parent;
+        // The alignment of the spill section may have diverged from the
+        // original, but correct assignment requires the spill's alignment,
+        // not the original.
+        isec->addralign = spill->addralign;
+
+        // Record the reduction in overage.
+        osec->memRegion->curPos -= isec->getSize();
----------------
mysterymath wrote:

That's true; this is definitely a best-effort satisficing type of algorithm.

The situation could be improved by running the spilling both forwards during assignment and backwards afterwards (as done here). This could be improved further binary searching for the earliest "split point" in the address assignment that allows the first memory region to fit, then fixing that moving on to the next overflowing region, etc, for `O(m log n)` time, where `m = |regions|` and `n = |sections|`. Alternatively, we may be able to handle alignment more specifically and accurately determine its effects. At the extreme end, we could get a kind of optimality by backtracking whenever a forward assignment overflows and performing the last spill opportunity, for `O(n^2)` time. That might not actually be too bad in practice.

Anyway, I do think that there's potential room to grow here, but my goal was to find the simplest algorithm that might satisfice most embedded projects. I had originally only had forward spilling (inline with assignment), but late unspillable sections cropped up too many times, and backwards spilling is resistant to them (while having its own concerns). 

https://github.com/llvm/llvm-project/pull/90007