[PATCH] D61666: [ELF] Optimize getISDThunkSec() to amortized O(1) in thunk creation

[Fangrui Song via Phabricator via llvm-commits]

MaskRay created this revision.
MaskRay added reviewers: ruiu, sfertile.
Herald added subscribers: llvm-commits, arichardson, emaste.
Herald added a reviewer: espindola.
Herald added a project: LLVM.

The time complexity of getISDThunkSec() is O(|ISD->ThunkSections|).
When thunk section spacing is enabled, ISD->ThunkSections can be in the
scale of size(ISD)/getThunkSectionSpacing(). In one large powerpc64le
executable, this number for .text is 25980.

It is very slow if every created thunk in the current
InputSectionDescription goes through the whole array.

Sort relocations by offset and leverage this property to optimize it to O(1):

If a thunk is located before the current relocation offset, once it
can't be reached, any subsequent relocation can't reach this thunk,
either.


Repository:
  rLLD LLVM Linker

https://reviews.llvm.org/D61666

Files:
  ELF/Relocations.cpp
  ELF/Relocations.h


Index: ELF/Relocations.h
===================================================================
--- ELF/Relocations.h
+++ ELF/Relocations.h
@@ -129,7 +129,7 @@
 
   ThunkSection *getISDThunkSec(OutputSection *OS, InputSection *IS,
                                InputSectionDescription *ISD, uint32_t Type,
-                               uint64_t Src);
+                               uint64_t Src, unsigned &ThunkSecIdx);
 
   ThunkSection *getISThunkSec(InputSection *IS);
 
Index: ELF/Relocations.cpp
===================================================================
--- ELF/Relocations.cpp
+++ ELF/Relocations.cpp
@@ -1240,10 +1240,9 @@
   for (auto I = Rels.begin(), End = Rels.end(); I != End;)
     scanReloc<ELFT>(Sec, GetOffset, I, End);
 
-  // Sort relocations by offset for more efficient searching for
-  // R_RISCV_PCREL_HI20 and R_PPC64_ADDR64.
-  if (Config->EMachine == EM_RISCV ||
-      (Config->EMachine == EM_PPC64 && Sec.Name == ".toc"))
+  // Efficient searching for R_RISCV_PCREL_HI20 and R_PPC64_ADDR64, and targets
+  // need thunks require relocations sorted by offset.
+  if (Target->NeedsThunks || Config->EMachine == EM_RISCV)
     llvm::stable_sort(Sec.Relocations,
                       [](const Relocation &LHS, const Relocation &RHS) {
                         return LHS.Offset < RHS.Offset;
@@ -1441,13 +1440,20 @@
 // linker script section pattern such as { .text .text.* }.
 ThunkSection *ThunkCreator::getISDThunkSec(OutputSection *OS, InputSection *IS,
                                            InputSectionDescription *ISD,
-                                           uint32_t Type, uint64_t Src) {
-  for (std::pair<ThunkSection *, uint32_t> TP : ISD->ThunkSections) {
-    ThunkSection *TS = TP.first;
+                                           uint32_t Type, uint64_t Src,
+                                           unsigned &ThunkSecIdx) {
+  for (unsigned E = ISD->ThunkSections.size(); ThunkSecIdx != E;
+       ++ThunkSecIdx) {
+    ThunkSection *TS = ISD->ThunkSections[ThunkSecIdx].first;
     uint64_t TSBase = OS->Addr + TS->OutSecOff;
     uint64_t TSLimit = TSBase + TS->getSize();
     if (Target->inBranchRange(Type, Src, (Src > TSLimit) ? TSBase : TSLimit))
       return TS;
+    if (Src < TSLimit)
+      break;
+
+    // If we reach here, we know ThunkSecIdx cannot satisfy any subsequent
+    // relocation, which has a larger offset. Thus we skip it (++ThunkSecIdx).
   }
 
   // No suitable ThunkSection exists. This can happen when there is a branch
@@ -1634,6 +1640,7 @@
   // InputSectionDescription as the caller.
   forEachInputSectionDescription(
       OutputSections, [&](OutputSection *OS, InputSectionDescription *ISD) {
+        unsigned ThunkSecIdx = 0;
         for (InputSection *IS : ISD->Sections)
           for (Relocation &Rel : IS->Relocations) {
             uint64_t Src = IS->getVA(Rel.Offset);
@@ -1658,7 +1665,7 @@
               if (auto *TIS = T->getTargetInputSection())
                 TS = getISThunkSec(TIS);
               else
-                TS = getISDThunkSec(OS, IS, ISD, Rel.Type, Src);
+                TS = getISDThunkSec(OS, IS, ISD, Rel.Type, Src, ThunkSecIdx);
               TS->addThunk(T);
               Thunks[T->getThunkTargetSym()] = T;
             }


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D61666.198595.patch
Type: text/x-patch
Size: 3273 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190508/71b13036/attachment-0001.bin>