[PATCH] D133003: [ELF] Parallelize relocation scanning

[Fangrui Song via Phabricator via llvm-commits]

MaskRay marked 3 inline comments as done.
MaskRay added a comment.

In D133003#3772446 <https://reviews.llvm.org/D133003#3772446>, @andrewng wrote:

>> Don't yet know the reason for the slow down but I suspect it will be related to the "size" of the tasks being spawned in parallel.
>
> Had some time to investigate a bit more and it seems that the slow down, at least on my 12C/24T Windows PC, is actually a result of contention over `relocMutex` in `RelocationScanner::processAux`. So "too many" concurrent threads running `RelocationScanner::processAux` can result in an overall slow down to scan the relocations and in these cases, it's likely to slow down even further with more available threads. Unfortunately, there's no mechanism in `parallel::TaskGroup` to limit the number of concurrent tasks being run by the pool from the group, so there's no "easy" solution. I've been experimenting with some ideas that shard the input sections such that there are fewer concurrent threads running the relocation scanning code.

Thanks for catching the issue. Perhaps we can add a `thread_local` thread index (for `getDefaultExecutor`) to llvm/Support/Parallel.h and allocate a relocation vector for each thread.



================
Comment at: lld/ELF/Relocations.cpp:1569
 
-static bool handleNonPreemptibleIfunc(Symbol &sym) {
+static bool handleNonPreemptibleIfunc(Symbol &sym, uint8_t flags) {
   // Handle a reference to a non-preemptible ifunc. These are special in a
----------------
ikudrin wrote:
> `uint8_t` -> `uint16_t`; not that it changes anything because the only flag that exceeds the range is `NEEDS_TLSIE` which is not used here, but still.
Thanks for catching this!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D133003/new/

https://reviews.llvm.org/D133003