[llvm] [DWARFLinker] Release input DWARF after object has been linked (PR #68376)

[Jonas Devlieghere via llvm-commits]

https://github.com/JDevlieghere created https://github.com/llvm/llvm-project/pull/68376

dsymutil is using an excessive amount of memory because it's holding on to the DWARF Context, even after it's done processing the corresponding object file. This patch releases the input DWARF after cloning, at which point it is no longer needed. This has always been the intended behavior, though I didn't bisect to figure out when this regressed.

When linking swift, this reduces peak (dirty) memory usage from 25 to 15 gigabytes.

rdar://111525100

>From 184153e472faeab973ef509e7d4c6b63ce46e43f Mon Sep 17 00:00:00 2001
From: Jonas Devlieghere <jonas at devlieghere.com>
Date: Thu, 5 Oct 2023 19:46:14 -0700
Subject: [PATCH] [DWARFLinker] Release input DWARF after object has been
 linked

dsymutil is using an excessive amount of memory because it's holding on
to the DWARF Context, even after it's done processing the corresponding
object file. This patch releases the input DWARF after cloning, at which
point it is no longer needed. This has always been the intended
behavior, though I didn't bisect to figure out when this regressed.

When linking swift, this reduces peak (dirty) memory usage from 25 to 15
gigabytes.

rdar://111525100
---
 llvm/include/llvm/DWARFLinker/DWARFLinker.h | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/llvm/include/llvm/DWARFLinker/DWARFLinker.h b/llvm/include/llvm/DWARFLinker/DWARFLinker.h
index 8489538e9b9f7a1..e5797514165a22b 100644
--- a/llvm/include/llvm/DWARFLinker/DWARFLinker.h
+++ b/llvm/include/llvm/DWARFLinker/DWARFLinker.h
@@ -269,9 +269,10 @@ class DWARFFile {
 public:
   using UnloadCallbackTy = std::function<void(StringRef FileName)>;
   DWARFFile(StringRef Name, std::unique_ptr<DWARFContext> Dwarf,
-            std::unique_ptr<AddressesMap> Addresses, UnloadCallbackTy = nullptr)
+            std::unique_ptr<AddressesMap> Addresses,
+            UnloadCallbackTy UnloadFunc = nullptr)
       : FileName(Name), Dwarf(std::move(Dwarf)),
-        Addresses(std::move(Addresses)) {}
+        Addresses(std::move(Addresses)), UnloadFunc(UnloadFunc) {}
 
   /// The object file name.
   StringRef FileName;
@@ -281,6 +282,18 @@ class DWARFFile {
 
   /// Helpful address information(list of valid address ranges, relocations).
   std::unique_ptr<AddressesMap> Addresses;
+
+  /// Callback to the module keeping object file to unload.
+  UnloadCallbackTy UnloadFunc;
+
+  /// Unloads object file and corresponding AddressesMap and Dwarf Context.
+  void unload() {
+    Addresses.reset();
+    Dwarf.reset();
+
+    if (UnloadFunc)
+      UnloadFunc(FileName);
+  }
 };
 
 typedef std::map<std::string, std::string> swiftInterfacesMap;
@@ -529,7 +542,8 @@ class DWARFLinker {
     /// the debug object.
     void clear() {
       CompileUnits.clear();
-      File.Addresses->clear();
+      ModuleUnits.clear();
+      File.unload();
     }
   };