[clang] [llvm] [clang][Modules][Serialization] Avoid serializing duplicate source location entries for shared input files (PR #209795)

[Ayokunle Amodu via cfe-commits]

================
@@ -1858,6 +1851,70 @@ ASTReader::readSLocOffset(ModuleFile *F, unsigned Index) {
   }
 }
 
+void ASTReader::buildLoadedInputFiles() {
+  LoadedInputFiles.emplace();
+  // Modules are visited in index order, which is fixed before this runs, so the
+  // copy chosen for a file is the same on every write of this module.
+  for (ModuleFile &F : ModuleMgr) {
+    for (unsigned I = 0, N = F.InputFilesLoaded.size(); I != N; ++I) {
+      InputFileInfo FI = getInputFileInfo(F, I + 1);
+      if (FI.UnresolvedImportedFilename.empty())
+        continue;
+      // An overridden input holds a buffer rather than the file named by its
+      // path, so its path and size cannot identify matching contents.
+      if (FI.Overridden)
+        continue;
+      (*LoadedInputFiles)[FI.StoredSize].push_back({&F, I + 1});
+    }
+  }
+}
+
+InputFileLoc ASTReader::getLoadedFileLoc(StringRef Path, off_t Size) {
+  if (!LoadedInputFiles)
+    buildLoadedInputFiles();
+
+  auto Known = LoadedInputFiles->find(Size);
+  if (Known == LoadedInputFiles->end())
+    return InputFileLoc();
+
+  StringRef WantedName = llvm::sys::path::filename(Path);
+  SmallString<128> Wanted;
+
+  for (const LoadedInputModuleFile &In : Known->second) {
+    InputFileInfo FI = getInputFileInfo(*In.F, In.InputID);
+
+    StringRef Unresolved = FI.UnresolvedImportedFilename;
+    if (llvm::sys::path::filename(Unresolved) != WantedName)
+      continue;
+
+    // Two directories can hold files that agree on name and on size, so the
+    // whole path decides.
+    if (Wanted.empty()) {
+      Wanted = Path;
+      FileMgr.makeAbsolutePath(Wanted, /*Canonicalize=*/true);
+    }
+    SmallString<128> Candidate;
+    {
+      auto Filename = ResolveImportedPath(PathBuf, Unresolved, *In.F);
+      Candidate = *Filename;
+    }
+    FileMgr.makeAbsolutePath(Candidate, /*Canonicalize=*/true);
+    if (StringRef(Candidate) != StringRef(Wanted))
+      continue;
----------------
ayokunle321 wrote:

I see `HeaderFileInfoTrait` checks size first, then mtime, and finally falls back to `FileEntry`.

I originally used `FileEntryRef` too, if you remember, but dropped it because resolving every file through `FileManager` caused unnecessary stat traffic. That was much earlier though when we were walking every file during module loading to find duplicates.

I then switched to name + size + mtime to avoid stat traffic, but dropped mtime per @rnk’s point, since the same file can have different mtimes across checkouts, copies, and filesystems.

And then that eventually evolved into this `makeAbsolutePath` approach you see here.

I think we can now follow the same approach as `HeaderFileInfoTrait`, minus mtime, falling back to `FileEntryRef` to handle the real-world files like symlinks and case-insensitive filesystems.

See [8098949](https://github.com/llvm/llvm-project/pull/209795/commits/80989491fd0d49d4d1d206f8c0ca29e49773d805) and let me know if you're happy with it.

https://github.com/llvm/llvm-project/pull/209795