[lld] [llvm] [DTLTO] Improve performance of adding files to the link (PR #186366)

[Ben Dunbobbin via llvm-commits]

================
@@ -2742,7 +2748,11 @@ class OutOfProcessThinBackend : public CGThinBackend {
           *(CacheStream.OS) << ObjFileMbRef.getBuffer();
           if (Error Err = CacheStream.commit())
             return Err;
+        } else if (AddBuffer) {
+          AddBuffer(Job.Task, Job.ModuleID, std::move(*ObjFileMbOrErr));
----------------
bd1976bris wrote:

Thanks. I have done the following now:
- Make AddBuffer mandatory for out-of-process backends
- Move AddStream from CGThinBackend to InProcessThinBackend
- Clarify linker comments to avoid implying the AddBuffer path is cache-specific

I also tried a follow-up refactor to move AddStreamFn from LTO::run() into
 createInProcessThinBackend() and make ThinBackend callbacks fully
 backend-specific. This required updating all in-process users (lld,
 llvm-lto2, gold, clang-nvlink-wrapper) and replacing overloaded
 constructors with factory helpers. However, I abandoned this because
 AddStreamFn is still needed by Full LTO via LTO::run(), so the change
 duplicated plumbing and increased complexity without clear benefit.

https://github.com/llvm/llvm-project/pull/186366