[lld] [llvm] [LLD][COFF] Prefetch inputs early-on to improve link times (PR #169224)
via llvm-commits
llvm-commits at lists.llvm.org
Sun Nov 23 08:12:48 PST 2025
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-lld
Author: Alexandre Ganea (aganea)
<details>
<summary>Changes</summary>
This PR reduces outliers in terms of runtime performance, by asking the OS to prefetch memory-mapped input files in advance, as early as possible. I have implemented the Linux aspect, however I have only tested this on Windows 11 version 24H2, with an active security stack enabled. The machine is a AMD Threadripper PRO 3975WX 32c/64t with 128 GB of RAM and Samsung 990 PRO SSD.
I have used a Unreal Engine-based game to profile the link times. Here's a quick summary of the input data:
```
Summary
--------------------------------------------------------------------------------
4,169 Input OBJ files (expanded from all cmd-line inputs)
26,325,429,114 Size of all consumed OBJ files (non-lazy), in bytes
9 PDB type server dependencies
0 Precomp OBJ dependencies
350,516,212 Input debug type records
18,146,407,324 Size of all input debug type records, in bytes
15,709,427 Merged TPI records
4,747,187 Merged IPI records
56,408 Output PDB strings
23,410,278 Global symbol records
45,482,231 Module symbol records
1,584,608 Public symbol records
```
In normal conditions - meanning all the pages are already in RAM - this PR has no noticeable effect:
```
>hyperfine "before\lld-link.exe @<!-- -->Game.exe.rsp" "with_pr\lld-link.exe @<!-- -->Game.exe.rsp"
Benchmark 1: before\lld-link.exe @<!-- -->Game.exe.rsp
Time (mean ± σ): 29.689 s ± 0.550 s [User: 259.873 s, System: 37.936 s]
Range (min … max): 29.026 s … 30.880 s 10 runs
Benchmark 2: with_pr\lld-link.exe @<!-- -->Game.exe.rsp
Time (mean ± σ): 29.594 s ± 0.342 s [User: 261.434 s, System: 62.259 s]
Range (min … max): 29.209 s … 30.171 s 10 runs
Summary
with_pr\lld-link.exe @<!-- -->Game.exe.rsp ran
1.00 ± 0.02 times faster than before\lld-link.exe @<!-- -->Game.exe.rsp
```
However when in production conditions, we're typically working with the Unreal Engine Editor, with exteral DCC tools like Maya, Houdini; we have several instances of Visual Studio open, VSCode with Rust analyzer, etc. All this means that between code change iterations, most of the input OBJs files might have been already evicted from the Windows RAM cache. Consequently, in the following test, I've simulated the worst case condition by evicting all data from RAM with [RAMMap64](https://learn.microsoft.com/en-us/sysinternals/downloads/rammap) (ie. `RAMMap64.exe -E[wsmt0]` with a 5-sec sleep at the end to ensure the System thread actually has time to evict the pages)
```
>hyperfine -p cleanup.bat "before\lld-link.exe @<!-- -->Game.exe.rsp" "with_pr\lld-link.exe @<!-- -->Game.exe.rsp"
Benchmark 1: before\lld-link.exe @<!-- -->Game.exe.rsp
Time (mean ± σ): 48.124 s ± 1.770 s [User: 269.031 s, System: 41.769 s]
Range (min … max): 46.023 s … 50.388 s 10 runs
Benchmark 2: with_pr\lld-link.exe @<!-- -->Game.exe.rsp
Time (mean ± σ): 34.192 s ± 0.478 s [User: 263.620 s, System: 40.991 s]
Range (min … max): 33.550 s … 34.916 s 10 runs
Summary
with_pr\lld-link.exe @<!-- -->Game.exe.rsp ran
1.41 ± 0.06 times faster than before\lld-link.exe @<!-- -->Game.exe.rsp
```
This is similar to the work done in MachO in https://github.com/llvm/llvm-project/pull/157917
---
Full diff: https://github.com/llvm/llvm-project/pull/169224.diff
6 Files Affected:
- (modified) lld/COFF/Driver.cpp (+7-1)
- (modified) llvm/include/llvm/Support/FileSystem.h (+2)
- (modified) llvm/include/llvm/Support/MemoryBuffer.h (+5)
- (modified) llvm/lib/Support/MemoryBuffer.cpp (+1)
- (modified) llvm/lib/Support/Unix/Path.inc (+13)
- (modified) llvm/lib/Support/Windows/Path.inc (+26)
``````````diff
diff --git a/lld/COFF/Driver.cpp b/lld/COFF/Driver.cpp
index 0e528de9c3652..e6e7194a684c1 100644
--- a/lld/COFF/Driver.cpp
+++ b/lld/COFF/Driver.cpp
@@ -164,6 +164,8 @@ static std::future<MBErrPair> createFutureForFile(std::string path) {
/*RequiresNullTerminator=*/false);
if (!mbOrErr)
return MBErrPair{nullptr, mbOrErr.getError()};
+ // Prefetch memory pages in the background as we will need them soon enough.
+ (*mbOrErr)->willNeedIfMmap();
return MBErrPair{std::move(*mbOrErr), std::error_code()};
});
}
@@ -337,8 +339,12 @@ void LinkerDriver::enqueuePath(StringRef path, bool wholeArchive, bool lazy) {
auto retryMb = MemoryBuffer::getFile(*retryPath, /*IsText=*/false,
/*RequiresNullTerminator=*/false);
ec = retryMb.getError();
- if (!ec)
+ if (!ec) {
mb = std::move(*retryMb);
+ // Prefetch memory pages in the background as we will need them soon
+ // enough.
+ mb->willNeedIfMmap();
+ }
} else {
// We've already handled this file.
return;
diff --git a/llvm/include/llvm/Support/FileSystem.h b/llvm/include/llvm/Support/FileSystem.h
index cf2a8104ac813..547d732dc3053 100644
--- a/llvm/include/llvm/Support/FileSystem.h
+++ b/llvm/include/llvm/Support/FileSystem.h
@@ -1310,6 +1310,7 @@ class mapped_file_region {
LLVM_ABI void unmapImpl();
LLVM_ABI void dontNeedImpl();
+ LLVM_ABI void willNeedImpl();
LLVM_ABI std::error_code init(sys::fs::file_t FD, uint64_t Offset,
mapmode Mode);
@@ -1341,6 +1342,7 @@ class mapped_file_region {
copyFrom(mapped_file_region());
}
void dontNeed() { dontNeedImpl(); }
+ void willNeed() { willNeedImpl(); }
LLVM_ABI size_t size() const;
LLVM_ABI char *data() const;
diff --git a/llvm/include/llvm/Support/MemoryBuffer.h b/llvm/include/llvm/Support/MemoryBuffer.h
index f092c67265a31..9b0f1f0f7d34e 100644
--- a/llvm/include/llvm/Support/MemoryBuffer.h
+++ b/llvm/include/llvm/Support/MemoryBuffer.h
@@ -83,6 +83,11 @@ class LLVM_ABI MemoryBuffer {
/// function should not be called on a writable buffer.
virtual void dontNeedIfMmap() {}
+ /// Mark the buffer as to-be-used in a near future. This shall trigger OS
+ /// prefetching from the storage device and into memory, if possible.
+ /// This should be use purely as an read optimization.
+ virtual void willNeedIfMmap() {}
+
/// Open the specified file as a MemoryBuffer, returning a new MemoryBuffer
/// if successful, otherwise returning null.
///
diff --git a/llvm/lib/Support/MemoryBuffer.cpp b/llvm/lib/Support/MemoryBuffer.cpp
index 23b9f8c5790d2..3ef4cd9086b95 100644
--- a/llvm/lib/Support/MemoryBuffer.cpp
+++ b/llvm/lib/Support/MemoryBuffer.cpp
@@ -243,6 +243,7 @@ class MemoryBufferMMapFile : public MB {
}
void dontNeedIfMmap() override { MFR.dontNeed(); }
+ void willNeedIfMmap() override { MFR.willNeed(); }
};
} // namespace
diff --git a/llvm/lib/Support/Unix/Path.inc b/llvm/lib/Support/Unix/Path.inc
index 0d991ead72416..0937adcd1bc18 100644
--- a/llvm/lib/Support/Unix/Path.inc
+++ b/llvm/lib/Support/Unix/Path.inc
@@ -900,6 +900,19 @@ void mapped_file_region::dontNeedImpl() {
#endif
}
+void mapped_file_region::willNeedImpl() {
+ assert(Mode == mapped_file_region::readonly);
+ if (!Mapping)
+ return;
+#if defined(__MVS__) || defined(_AIX)
+ // If we don't have madvise, or it isn't beneficial, treat this as a no-op.
+#elif defined(POSIX_MADV_WILLNEED)
+ ::posix_madvise(Mapping, Size, POSIX_MADV_WILLNEED);
+#else
+ ::madvise(Mapping, Size, MADV_WILLNEED);
+#endif
+}
+
int mapped_file_region::alignment() { return Process::getPageSizeEstimate(); }
std::error_code detail::directory_iterator_construct(detail::DirIterState &it,
diff --git a/llvm/lib/Support/Windows/Path.inc b/llvm/lib/Support/Windows/Path.inc
index be007b7abdb51..8be4105ba6bd8 100644
--- a/llvm/lib/Support/Windows/Path.inc
+++ b/llvm/lib/Support/Windows/Path.inc
@@ -1023,6 +1023,32 @@ void mapped_file_region::unmapImpl() {
void mapped_file_region::dontNeedImpl() {}
+void mapped_file_region::willNeedImpl() {
+#if (_WIN32_WINNT < _WIN32_WINNT_WIN8)
+ typedef struct _WIN32_MEMORY_RANGE_ENTRY {
+ PVOID VirtualAddress;
+ SIZE_T NumberOfBytes;
+ } WIN32_MEMORY_RANGE_ENTRY, *PWIN32_MEMORY_RANGE_ENTRY;
+#endif
+
+ HMODULE kernelM = llvm::sys::windows::loadSystemModuleSecure(L"kernel32.dll");
+ if (kernelM) {
+ // PrefetchVirtualMemory is only available on Windows 8 and later. Since we
+ // still support compilation on Windows 7, we load the function dynamically.
+ typedef BOOL(WINAPI * PrefetchVirtualMemory_t)(
+ HANDLE hProcess, ULONG_PTR NumberOfEntries,
+ _In_reads_(NumberOfEntries) PWIN32_MEMORY_RANGE_ENTRY VirtualAddresses,
+ ULONG Flags);
+ static const auto pfnPrefetchVirtualMemory =
+ (PrefetchVirtualMemory_t)::GetProcAddress(kernelM,
+ "PrefetchVirtualMemory");
+ if (pfnPrefetchVirtualMemory) {
+ WIN32_MEMORY_RANGE_ENTRY Range{Mapping, Size};
+ pfnPrefetchVirtualMemory(::GetCurrentProcess(), 1, &Range, 0);
+ }
+ }
+}
+
std::error_code mapped_file_region::sync() const {
if (!::FlushViewOfFile(Mapping, Size))
return mapWindowsError(GetLastError());
``````````
</details>
https://github.com/llvm/llvm-project/pull/169224
More information about the llvm-commits
mailing list