[lld] [llvm] [DTLTO] Delay individual bitcode file serialization until after cache checks (PR #191142)

[Andrew Ng via llvm-commits]

================
@@ -2410,11 +2422,71 @@ class OutOfProcessThinBackend : public CGThinBackend {
   // Callback to add a pre-existing native object buffer to the link.
   AddBufferFn AddBuffer;
 
+  // Map from module ID to input buffer for modules whose module IDs were
+  // rewritten to distinct temporary filenames (e.g. archive members or
+  // FatLTO objects). This allows the ThinLTO backend to serialize those
+  // modules at the rewritten paths. Serialization is deferred until the
+  // backend determines that the compilation job is not cached.
+  const StringMap<MemoryBufferRef> &BitcodeForDistribution;
+
+  // Records which bitcode input files were seralized.
+  SmallVector<StringRef> SerializedBitcode;
+
+  // For DTLTO, some ThinLTO-enabled inputs may have their module IDs rewritten
+  // to distinct temporary filenames so the external backend compilation can
+  // reference an individual bitcode file. Serialize such modules for uncached
+  // backend jobs, along with any imported modules, even if those imports were
+  // satisfied from the cache.
+  Error serializeBitcodeForUncachedJobs() {
+    if (BitcodeForDistribution.empty())
+      return Error::success();
+
+    DenseSet<StringRef> Seen;
+
+    auto serializeBitcode = [&](StringRef ModuleID) -> Error {
+      auto It = BitcodeForDistribution.find(ModuleID);
+      if (It == BitcodeForDistribution.end() ||
+          !Seen.insert(It->getKey()).second)
----------------
nga888 wrote:

Instead of this additional "tracking" via `Seen`, perhaps erase `It` from `BitcodeForDistribution` instead? However, `BitcodeForDistribution` would need to be made non `const` for this.

https://github.com/llvm/llvm-project/pull/191142