[lld] [llvm] [LLD][COFF] Prefetch inputs early-on to improve link times (PR #169224)

[Alexandre Ganea via llvm-commits]

aganea wrote:

@hans I addressed your comments.

> When you say the system freezes for multiple seconds, does the build still end up faster with this flag, or does the freeze negate the benefit?

I investigated a bit more and this is mainly caused by `PrefetchVirtualMemory` which waits for the Standby List lock which is also taken by the storage drive which brings the pages asynchronously into RAM (into the Standby List).

<img width="1351" height="724" alt="Capture d’écran 2026-01-08 154713" src="https://github.com/user-attachments/assets/20920270-7226-420a-9e1d-868ef78c4cde" />

The OS is not frozen per se, it is greatly slowed down while the prefetching occurs (again, only if there are a large number of inputs to handle which do not all fit in RAM). And it is indeed detrimental to the overall link times when that occurs.

As an example, a Win64 target which takes approx 160 GB of input files (mostly .OBJs):
```
Before this PR: 432 sec
After this PR:  500 sec
```
Summary:
```
                                    Summary
--------------------------------------------------------------------------------
              20,575 Input OBJ files (expanded from all cmd-line inputs)
     162,743,684,034 Size of all consumed OBJ files (non-lazy), in bytes
                  47 PDB type server dependencies
                   0 Precomp OBJ dependencies
       2,076,402,219 Input debug type records
     122,154,522,140 Size of all input debug type records, in bytes
          14,755,332 Merged TPI records
           4,133,367 Merged IPI records
              47,350 Output PDB strings
         166,232,502 Global symbol records
         109,924,995 Module symbol records
           2,190,897 Public symbol records
```

This is an extreme case, which is worse than what is suggested in https://crbug.com/428641952. It is a non-jumbo target which we usually don't compile, only occassionaly to validate that we have `#include`d indeed all necessary headers in .CPP files. In jumbo mode it happens often that prior .CPPs included in the jumbo .CPP are implicity using `#include`s from prior files. So it can take a while until we see dependency issues.

Another target which has some level of caching does not have this problem:
```
Before or after this PR: 242 sec
```
Summary:
```
                                    Summary
--------------------------------------------------------------------------------
              17,061 Input OBJ files (expanded from all cmd-line inputs)
     120,979,726,253 Size of all consumed OBJ files (non-lazy), in bytes
                   5 PDB type server dependencies
                   0 Precomp OBJ dependencies
       1,577,138,629 Input debug type records
      91,241,482,224 Size of all input debug type records, in bytes
          11,626,762 Merged TPI records
           3,092,110 Merged IPI records
              38,873 Output PDB strings
         126,437,475 Global symbol records
          69,627,439 Module symbol records
           1,334,901 Public symbol records
```

The very same target as compiled in production with jumbo files (same build flags):
```
                                    Summary
--------------------------------------------------------------------------------
               3,667 Input OBJ files (expanded from all cmd-line inputs)
      23,001,559,157 Size of all consumed OBJ files (non-lazy), in bytes
                   5 PDB type server dependencies
                   0 Precomp OBJ dependencies
         252,435,120 Input debug type records
      14,894,115,232 Size of all input debug type records, in bytes
          11,601,734 Merged TPI records
           3,039,405 Merged IPI records
              38,998 Output PDB strings
          20,996,737 Global symbol records
          66,433,008 Module symbol records
           1,327,982 Public symbol records
```

The total input files are about an order of magnitude smaller, and so are the link times:
```
                HOT cache   COLD cache
Before this PR:   17 sec      36 sec
After this PR:    17 sec      21 sec
```


https://github.com/llvm/llvm-project/pull/169224