[PATCH] D96035: [WIP][dsymutil][DWARFlinker] implement separate multi-thread processing for compile units.

[Greg Clayton via Phabricator via llvm-commits]

clayborg added a comment.

In D96035#2585694 <https://reviews.llvm.org/D96035#2585694>, @friss wrote:

> In D96035#2585324 <https://reviews.llvm.org/D96035#2585324>, @avl wrote:
>
>>> How would you guarantee that the artificial CU is always emitted in the same order? It seems like you need to stash the types somewhere, then create a deterministic ordering and then emit it, but I don't remember whether that's doable with LLVM's DIEs (It's been a while since I touched this code).
>>
>> We might sort types on name basis inside that artificial CU. In this case it would always be emitted in the same order.
>
> I don't think finding how to sort them is the complicated part. In my mind the issue is keeping them around until you can sort them as I'm not sure DIEs can exist not attached to a CU. Also, it's not just the one type, but the transitive closure of all its dependencies that you need to consider which makes things (much) more complicated. As I said before, I haven't worked on this in a long time, so feel free to shoot my concerns down!

I agree with Fred in that it might be too expensive to keep all of the CUs around so that we can later copy type data out of them at the end into uniqued compile units.

For testing DWARF, a while back I created a DWARF generator set of classes:

llvm/unittests/DebugInfo/DWARF/DwarfGenerator.h
llvm/unittests/DebugInfo/DWARF/DwarfGenerator.cpp

This allows the creation of stand alone DWARF using a new set of DWARF classes:

llvm::dwarfgen::DIE
llvm::dwarfgen::CompileUnit
llvm::dwarfgen::Generator

It might allow us to copy the types into a llvm::dwarfgen::Generator instance and might easy the ability to add and remove DIEs to parent DIEs, like adding new methods to classes and slowly adding new types a compile unit DIE. Check the test files in llvm/unittests/DebugInfo/DWARF that include the DwarfGenerator.h file to see examples of tests that create DWARF using this generator. The nice thing is all references to other DIEs are actual pointers to llvm::dwarfgen::DIE objects, so you can insert and remove DIEs as needed to anything within a llvm::dwarfgen::CompileUnit, even across llvm::dwarfgen::CompileUnit objects if needed, and when you generate the DWARF, it all gets finalized at that point. So maybe we can use the DWARF generator for all types. When each compile unit is done being parsed on its thread, it can enter a type copying phase where it makes sure all of the uniqued types it needs are copied into a single smart class that uses the DWARF generator. Then we can unload the compile units from the .o files right away when we are done.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D96035/new/

https://reviews.llvm.org/D96035