[all-commits] [llvm/llvm-project] 2b2428: [lld][MachO] Multi-threaded preload of input files...

[John Holdsworth via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 2b2428794c0fc86a3eed74a264f2dd0a9548487c
      https://github.com/llvm/llvm-project/commit/2b2428794c0fc86a3eed74a264f2dd0a9548487c
  Author: John Holdsworth <github at johnholdsworth.com>
  Date:   2025-08-27 (Wed, 27 Aug 2025)

  Changed paths:
    M lld/MachO/Config.h
    M lld/MachO/Driver.cpp
    M lld/MachO/Options.td

  Log Message:
  -----------
  [lld][MachO] Multi-threaded preload of input files into memory (#147134)

This PR adds a new option to lld `--read-workers=20` that defers all
disk I/o then performs it multithreaded so the process is never stalled
waiting for the I/o of the page-in of mapped input files. This results
in a saving of elapsed time. For a large link (iterating on Chromium)
these are the baseline linkage times saving a single file and rebuilding
(seconds inside Xcode):

26.01, 25.84, 26.15, 26.03, 27.10, 25.90, 25.86, 25.81, 25.80, 25.87

With the proposed code change, and using the `--read-workers=20` option,
the linking times reduce to the following:

21.13, 20.35, 20.01, 20.01, 20.30, 20.39, 19.97, 20.23, 20.17, 20.23

The secret sauce is in the new function `multiThreadedPageIn()` in
Driver.cpp. Without the option lld behaves as before.

Edit: with subsequent commits I've taken this novel i/o approach to its
full potential. Latest linking times are now:

13.2, 11.9, 12.12, 12.01, 11.99, 13.11, 11.93, 11.95, 12.18, 11.97

Chrome is still linking and running so it doesn't look like anything is
broken. Despite being multi-threaded all memory access is readonly and
the original code paths are not changed. All that is happening is the
system is being asked to proactively page in files rather than waiting
for processing to page fault which would otherwise stall the process.

---------

Co-authored-by: Daniel Rodríguez Troitiño <drodrigueztroitino at gmail.com>
Co-authored-by: Ellis Hoag <ellis.sparky.hoag at gmail.com>



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications