[llvm] 29a1567 - [DWARFLinker] Release input DWARF after object has been linked (#68376)

[via llvm-commits]

Author: Jonas Devlieghere
Date: 2023-10-06T08:00:48-07:00
New Revision: 29a1567435ed56a56410fce76d0dedc89683dc81

URL: https://github.com/llvm/llvm-project/commit/29a1567435ed56a56410fce76d0dedc89683dc81
DIFF: https://github.com/llvm/llvm-project/commit/29a1567435ed56a56410fce76d0dedc89683dc81.diff

LOG: [DWARFLinker] Release input DWARF after object has been linked (#68376)

dsymutil is using an excessive amount of memory because it's holding on
to the DWARF Context, even after it's done processing the corresponding
object file. This patch releases the input DWARF after cloning, at which
point it is no longer needed. This has always been the intended
behavior, though I didn't bisect to figure out when this regressed.

When linking swift, this reduces peak (dirty) memory usage from 25 to 15
gigabytes.

rdar://111525100

Added: 
    

Modified: 
    llvm/include/llvm/DWARFLinker/DWARFLinker.h

Removed: 
    


################################################################################
diff  --git a/llvm/include/llvm/DWARFLinker/DWARFLinker.h b/llvm/include/llvm/DWARFLinker/DWARFLinker.h
index 8489538e9b9f7a1..e5797514165a22b 100644
--- a/llvm/include/llvm/DWARFLinker/DWARFLinker.h
+++ b/llvm/include/llvm/DWARFLinker/DWARFLinker.h
@@ -269,9 +269,10 @@ class DWARFFile {
 public:
   using UnloadCallbackTy = std::function<void(StringRef FileName)>;
   DWARFFile(StringRef Name, std::unique_ptr<DWARFContext> Dwarf,
-            std::unique_ptr<AddressesMap> Addresses, UnloadCallbackTy = nullptr)
+            std::unique_ptr<AddressesMap> Addresses,
+            UnloadCallbackTy UnloadFunc = nullptr)
       : FileName(Name), Dwarf(std::move(Dwarf)),
-        Addresses(std::move(Addresses)) {}
+        Addresses(std::move(Addresses)), UnloadFunc(UnloadFunc) {}
 
   /// The object file name.
   StringRef FileName;
@@ -281,6 +282,18 @@ class DWARFFile {
 
   /// Helpful address information(list of valid address ranges, relocations).
   std::unique_ptr<AddressesMap> Addresses;
+
+  /// Callback to the module keeping object file to unload.
+  UnloadCallbackTy UnloadFunc;
+
+  /// Unloads object file and corresponding AddressesMap and Dwarf Context.
+  void unload() {
+    Addresses.reset();
+    Dwarf.reset();
+
+    if (UnloadFunc)
+      UnloadFunc(FileName);
+  }
 };
 
 typedef std::map<std::string, std::string> swiftInterfacesMap;
@@ -529,7 +542,8 @@ class DWARFLinker {
     /// the debug object.
     void clear() {
       CompileUnits.clear();
-      File.Addresses->clear();
+      ModuleUnits.clear();
+      File.unload();
     }
   };