[PATCH] D96035: [dsymutil][DWARFlinker] implement separate multi-thread processing for compile units.

[David Blaikie via Phabricator via llvm-commits]

dblaikie added a comment.

> These imported declarations amounted to 3% of overall CU size for a single compile unit that I tested. I know that the stock dsymutil whitelists all of these which causes these and all the types the reference to be kept, so if we can get rid of any DW_TAG_imported_declaration DIEs that are not referenced, we might also be able to get rid of the type that they reference as well as all types from a C++ header file that have corresponding C header files like "cstdlib" -> "stdlib.h", they will have this issue and waste space.

This is a more fundamental issue with Clang's debug info (I say this as the person who implemented the support for imported declarations in Clang) - Clang doesn't track the use of a using decl (& for a using directive, that brings in a whole namespace, there's no DWARF that would reference that anyway (eg: a variable is of some type - so you have to refer to the type DIE, there's nowhere to mention/reference an imported namespace from that association)) so Clang's debug info describes the type referenced via a using decl in the source without using the using decl in the DWARF:

  $ cat decl.cpp
  namespace ns {
  struct t1 { };
  }
  using ns::t1;
  t1 v1;
  $ clang++-tot decl.cpp -g -c && llvm-dwarfdump-tot decl.o | grep "DW_AT_type\|DW_TAG"
  0x0000000b: DW_TAG_compile_unit
  0x0000001e:   DW_TAG_variable
                  DW_AT_type      (0x00000038 "ns::t1")
  0x00000033:   DW_TAG_namespace
  0x00000038:     DW_TAG_structure_type
  0x00000042:   DW_TAG_imported_declaration

So imported modules are never referenced from other DWARF - if you want any of them to persist you have to treat them all as roots, you can't GC them.
And for imported declarations, at least those generated from Clang you have to do the same until that bug is fixed in Clang to actually reference the using declarations.

In any case I assume this issue is mostly orthogonal to this proposal (ie: I expect that's an existing issue in dsymutil, not a new regression introduced by this patch), yeah?

In D96035#3221098 <https://reviews.llvm.org/D96035#3221098>, @clayborg wrote:

> One issue I do see with this approach as well is the first "__type_table" is about 69MB for a dSYM file for LLDB.framework in LLDB. Any types that are accessed from debug info in LLDB will end up causing this entire DW_TAG_compile_unit to be parsed which can be expensive as LLDB parses DWARF partially and only parses what it needs to when loading DWARF. Is there any way to split up the "__type_table" into multiple DW_TAG_compile_unit entries whose DW_AT_name matches the DW_AT_decl_file of the root type that is being uniqued? That would allow smaller compile units to be parsed and keep memory footprint down in debuggers.

Splitting them up seems OK to me. Though there's some tradeoff - encodings can be more dense if clusters of types are in the same unit (since they can use shorter unit-local references, rather than full sec_offset references) and th extra unit headers. Probably groups of types in units would be good - a basic heuristic would be X types per unit, but a more advanced one would depend on the size of the types (& even more advanced would group related types together to benefit from more local references - but I doubt that's worth it).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D96035/new/

https://reviews.llvm.org/D96035