[llvm] [DWARFLinkerParallel] Add support for -odr mode. (PR #68721)

[via llvm-commits]

avl-llvm wrote:

> Why is the debug info section so much smaller when using the parallel linker? Is it really that much more efficient than the Apple one, and if so, where are the gains coming from? Are we eliminating types that the dsymutil isn't, like base types? I'm a little skeptical, but I can also imagine the new approach is a lot better because it doesn't need to keep things like forward declarations until it sees a definition, etc.

yes, the size of debug info is smaller because the parallel linker does ODR deduplication differently:

1. The most space-saving thing is types merging(*). Parallel linker creates
an artificial compile unit and moves all types into this unit. All variations
of partially defined type are finally combined into the single type description.
This saves most of the space. f.e.

llvm-xray built with llvm linker:
```
llvm-dwarfdump --debug-info llvm-xray.dSYM/Contents/Resources/DWARF/llvm-xray | grep -c "\(\"vector<unsigned char, std::__1::allocator<unsigned char> >\"\)"
1

```
llvm-xray built with apple linker:
```
llvm-dwarfdump --debug-info llvm-xray.dSYM/Contents/Resources/DWARF/llvm-xray | grep -c "\(\"vector<unsigned char, std::__1::allocator<unsigned char> >\"\)"
140

```
2. Forward declarations are removed. The type(if the definition is found) or single type declaration
is moved into the artificial compile unit. All other copies of the declaration are removed.

3. Not used content of imported namespaces is removed.


> 
> > Note: Sometimes DWARFLinkerParallel may produce non-deterministic results.
> 
> I know you're aware but I'll emphasize again that that's a non-starter for us. It sounds like you have patches in the works so that's great!

yes, I intend to have deterministic output. I think it would be better to achieve this goal with separate patches as the current patch is already huge. The problem that leads to non-deterministic output is caused by inconsistent DWARF(**). It contains different definitions for the same type. So the definition that would be put in the result depends on which definition was handled earlier(which is non-deterministic). I think the proper way to fix that problem would be to fix Clang's DWARF generation to produce consistent results. 


(*) Example of types merging:

Lets assume we have following types:
```

CU1 {
  DW_TAG_class_type {
      DW_TAG_name "vector"
      DW_AT_declaration (true)
      
      DW_TAG_subprogram {
          DW_TAG_name "push_back"
      }
  }
  
 DW_TAG_typedef
   DW_AT_Name "vector"
   DW_AT_type CU1::vector
}

CU2 {
  DW_TAG_class_type {
      DW_TAG_name "vector"
      DW_AT_declaration (true)
      
      DW_TAG_subprogram {
          DW_TAG_name "empty"
      }
  }

  
 DW_TAG_typedef
   DW_AT_Name "vector"
   DW_AT_type CU2::vector  
}

```
Above partial definitions would be united into single one:

```
TYPE_CU {
  DW_TAG_class_type {
      DW_TAG_name "vector"
      DW_AT_declaration (true)
      
      DW_TAG_subprogram {
          DW_TAG_name "push_back"
      }
      DW_TAG_subprogram {
          DW_TAG_name "empty"
      }      
  }
}

CU1 {
 DW_TAG_typedef
   DW_AT_Name "vector"
   DW_AT_type TYPE_CU::vector
}

CU2 { 
 DW_TAG_typedef
   DW_AT_Name "vector"
   DW_AT_type TYPE_CU::vector  
}

```
(**) Example of inconsistent DWARF:

1. opt binary.

compile unit "llvm/tools/opt/opt.cpp" has following definition:

```
0x001c4067:     DW_TAG_class_type
                  DW_AT_calling_convention      (DW_CC_pass_by_reference)
                  DW_AT_name    ("DominatorTreeBase<llvm::BasicBlock, false>")
                  DW_AT_byte_size       (0x50)
                  DW_AT_decl_file       ("llvm-project/llvm/include/llvm/IR/Dominators.h")
                  DW_AT_decl_line       (51)

0x001c4090:       DW_TAG_member
                    DW_AT_name  ("Insert")
                    DW_AT_type  (0x002aa3e4 "const UpdateKind")
                    DW_AT_decl_file     ("llvm-project/llvm/include/llvm/Support/GenericDomTree.h")
                    DW_AT_decl_line     (258)
                    DW_AT_external      (true)
                    DW_AT_declaration   (true)
                    DW_AT_accessibility (DW_ACCESS_public)

```
while compile unit "llvm/lib/Target/X86/X86LowerAMXIntrinsics.cpp" has a bit different:

```
0x010f6214:     DW_TAG_class_type
                  DW_AT_calling_convention      (DW_CC_pass_by_reference)
                  DW_AT_name    ("DominatorTreeBase<llvm::BasicBlock, false>")
                  DW_AT_byte_size       (0x50)
                  DW_AT_decl_file       ("llvm-project/llvm/include/llvm/IR/Dominators.h")
                  DW_AT_decl_line       (51)

0x010f623d:       DW_TAG_member
                    DW_AT_name  ("Insert")
                    DW_AT_type  (0x011774c9 "const UpdateKind")
                    DW_AT_decl_file     ("llvm-project/llvm/include/llvm/Support/GenericDomTree.h")
                    DW_AT_decl_line     (258)
                    DW_AT_external      (true)
                    DW_AT_declaration   (true)
                    DW_AT_accessibility (DW_ACCESS_public)
                    DW_AT_const_value   (0)   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<

```

2. llvm-xray binary.

compile unit "llvm/lib/Bitcode/Reader/BitcodeReader.cpp" has following definition:

```
0x0260e77a:     DW_TAG_class_type
                  DW_AT_calling_convention      (DW_CC_pass_by_reference)
                  DW_AT_name    ("SmallVectorImpl<llvm::Instruction *>")
                  DW_AT_byte_size       (0x10)
                  DW_AT_decl_file       ("llvm-project/llvm/include/llvm/ADT/SmallVector.h")
                  DW_AT_decl_line       (577)

0x0260e784:       DW_TAG_template_type_parameter
                    DW_AT_type  (0x026cb21f "llvm::Instruction *")
                    DW_AT_name  ("T")   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

```
while compile unit "llvm/lib/IR/SSAContext.cpp" has a bit different:

```
0x03659f68:     DW_TAG_class_type
                  DW_AT_calling_convention      (DW_CC_pass_by_reference)
                  DW_AT_name    ("SmallVectorImpl<llvm::Instruction *>")
                  DW_AT_byte_size       (0x10)
                  DW_AT_decl_file       ("llvm-project/llvm/include/llvm/ADT/SmallVector.h")
                  DW_AT_decl_line       (577)

0x03659f72:       DW_TAG_template_type_parameter
                    DW_AT_type  (0x036716fa "llvm::Instruction *")
```

https://github.com/llvm/llvm-project/pull/68721