[clang] [llvm] [clang][Modules][Serialization] Avoid serializing duplicate source location entries for shared input files (PR #209795)

[Ayokunle Amodu via cfe-commits]

ayokunle321 wrote:

> So it seems we are spending a lot of time calling `makeAbsolute`. Maybe we can make `LoadedInputFiles` take base name, or a base name + file size pair as key so we can avoid calling `makeAbsolute` on all the input files?

> Another direction to explore can be making the logic in `buildLoadedInputFiles()` lazy. Currently, the amount of work it performs depends on the size of the imported modules. I think it's usually better to make it proportional to the size of the written module itself. (That holds true especially for large module graphs.) Why don't we use the same visitor+lookup pattern we already use for `HeaderFileInfo` here?

@jansvoboda11 @qiongsiwu

Looked into this. For the SLoc scan, I changed the `INPUT_FILE` record to carry the SLoc index and offset directly, captured in `WriteInputFiles` where we already walk the SLoc entries. So the reader reconstructs the location from those instead of scanning, which let me delete the per-module SLoc scan entirely. It adds ~322 B per PCM for my workload, which is minimal but is not nothing since we're wary about PCM size.  It does reduce the complexity and a net deletion in code overall.

For the outer lookup which asks "which loaded module holds the file", I keyed the map on size instead of the canonicalized path, so we skip `makeAbsolutePath` on every input file and only fall back to full path on the candidates in the same size bucket. I went with size alone because the string map inserts were ~20% of the added cost in @qiongsiwu's run, and a size key drops the string hashing entirely. Collisions should be rare, and full path still confirms identity, but let me know if you'd rather I key on name + size for the initial check. 

On a workload (generated my Claude) shaped like your SDK TU (212 modules, SDK-length paths) this removed ~77% of the added cost, which lines up with the ~77% your instrumentation put on path canonicalization. So the eager build is cheap now even though it still walks the loaded graph.

On making `buildLoadedInputFiles()` actually lazy, from the visitor pattern it looks like each module would need to serialize its own input-file table into the PCM to probe instead of building the map at load time. I prototyped that and measured it and the table adds ~2 KB per PCM (~0.42%) to recover ~1.5% of the time regression (this is after the `INPUT_FILE` record change and the scan removal), and it's written to every PCM including leaf modules that nothing is ever written against. So I was hesitant to go ahead with it.

The O(written module) lazy version is definitely the better design and it would cut code complexity too. But the size-keying already made the eager build cheap enough that I'm not sure the table earns the ~2 KB (not sure what the threshold is for accepted PCM size increases).

My numbers are from synthetic workloads, so if @qiongsiwu could re-measure timing that would be great!

https://github.com/llvm/llvm-project/pull/209795