[lld] [lld][MachO]Multi-threaded i/o. Twice as fast linking a large project. (PR #147134)

[Daniel Rodríguez Troitiño via llvm-commits]

================
@@ -282,11 +284,87 @@ static void saveThinArchiveToRepro(ArchiveFile const *file) {
           ": Archive::children failed: " + toString(std::move(e)));
 }
 
-static InputFile *addFile(StringRef path, LoadType loadType,
-                          bool isLazy = false, bool isExplicit = true,
-                          bool isBundleLoader = false,
-                          bool isForceHidden = false) {
-  std::optional<MemoryBufferRef> buffer = readFile(path);
+class DeferredFile {
+public:
+  StringRef path;
+  bool isLazy;
+  MemoryBufferRef buffer;
+};
+using DeferredFiles = std::vector<DeferredFile>;
+
+// Most input files have been mapped but not yet paged in.
+// This code forces the page-ins on multiple threads so
+// the process is not stalled waiting on disk buffer i/o.
+void multiThreadedPageInBackground(const DeferredFiles &deferred) {
+  static size_t pageSize = Process::getPageSizeEstimate(), totalBytes;
+  static std::mutex mutex;
+  size_t index = 0;
+
+  parallelFor(0, config->readThreads, [&](size_t I) {
+    while (true) {
+      mutex.lock();
+      if (index >= deferred.size()) {
+        mutex.unlock();
+        return;
+      }
+      const StringRef &buff = deferred[index].buffer.getBuffer();
+      totalBytes += buff.size();
+      index += 1;
+      mutex.unlock();
+
+      // Reference each page to load it into memory.
+      for (const char *page = buff.data(), *end = page + buff.size();
+           page < end; page += pageSize)
+        volatile char t = *page;
+    }
+  });
+
+  if (getenv("LLD_MULTI_THREAD_PAGE"))
----------------
drodriguez wrote:

If `totalBytes` is only used in the debug builds, its code can be surrounded in the `LLVM_DEBUG()` macro to avoid those unused warnings.

```
#ifndef NDEBUG
static size_t totalBytes = 0;
#endif
...
LLVM_DEBUG(totalBytes += buff.size());
...
LLVM_DEBUG(
if (getenv("LLD_MULTI_THREAD_PAGE"))
  llvm::dbgs() << "multiThreadedPageIn " << totalBytes << "/"
                 << deferred.size() << "\n"
);
```

https://github.com/llvm/llvm-project/pull/147134