[llvm] [BOLT][DWARF][NFC] Change how we re-size CloneUnitCtxMap (PR #75876)

Amir Ayupov via llvm-commits llvm-commits at lists.llvm.org
Tue Dec 19 12:41:44 PST 2023


aaupov wrote:

Context: 
BOLT normally processes CUs in a way to reduce memory overhead because raising DWARF to IR is memory intensive.

All CUs are partitioned in the following way: all CUs with cross-references are put in a single bucket, and the rest are put into buckets of size `cu-processing-batch-size` (default is 1, meaning they are handled one by one). This partitioning is done in `partitionCUs`, which returns a vector of vector of DWARFUnit pointers, where each first-level vector entry is a partition. https://github.com/llvm/llvm-project/blob/78a195e1002dbfdfaeb7b36d5699e58b47238cbb/bolt/lib/Rewrite/DWARFRewriter.cpp#L568-L571

However, in `buildCompileUnits` invoked from `DWARFRewriter::updateDebugInfo` with those partitions, we used to allocate space in `CloneUnitCtxMap` for all CUs in `DwarfContext` (all CUs in the input binary):
https://github.com/llvm/llvm-project/blob/219355d4c0d2b6e2c0d5e022f8b7a78c1e9ce53f/bolt/lib/Core/DIEBuilder.cpp#L275

For large binaries this is overly conservative and caused massive memory allocations. It is enough to allocate space only for CUs in the current partition – either for current CU itself (no cross-CU refs), or for all CUs that have cross-refs (which are all put in the same partition).

Note that the partition assignment is based on the heuristic that CUs with cross-refs share the abbrev_table. If this is not the case, BOLT detects that using `DUList` so extra entries are allocated in `CloneUnitCtxMap` to ensure correctness.

https://github.com/llvm/llvm-project/pull/75876


More information about the llvm-commits mailing list