[all-commits] [llvm/llvm-project] 60cd3e: Reduce llvm-gsymutil memory usage (#91023)

[Kevin Frei via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 60cd3eb880fe48d192a58c64a1e38e875fc65377
      https://github.com/llvm/llvm-project/commit/60cd3eb880fe48d192a58c64a1e38e875fc65377
  Author: Kevin Frei <kevinfrei at users.noreply.github.com>
  Date:   2024-07-02 (Tue, 02 Jul 2024)

  Changed paths:
    M llvm/include/llvm/DebugInfo/DWARF/DWARFUnit.h
    M llvm/lib/DebugInfo/DWARF/DWARFUnit.cpp
    M llvm/lib/DebugInfo/GSYM/DwarfTransformer.cpp

  Log Message:
  -----------
  Reduce llvm-gsymutil memory usage (#91023)

llvm-gsymutil eats a lot of RAM. On some large binaries, it causes OOM's on smaller hardware, consuming well over 64GB of RAM. This change frees line tables once we're done with them, and frees DWARFUnits's DIE's when we finish processing each DU, though they may get reconstituted if there are references from other DU's during processing. Once the conversion is complete, all DIE's are freed. The reduction in peak memory usage from these changes showed between 7-12% in my tests.

The double-checked locking around the creation & freeing of the data structures was tested on a 166 core system. I validated that it trivially malfunctioned without the locks (and with stupid reordering of the locks) and worked reliably with them.

---------

Co-authored-by: Kevin Frei <freik at meta.com>



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications