[all-commits] [llvm/llvm-project] 460a22: [DWARF][BOLT] Implement new mechanism for DWARFRew...

Alexander Yermolovich via All-commits all-commits at lists.llvm.org
Thu Jul 6 14:21:45 PDT 2023 

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 460a2244430fae192298a5fd9fa2a269e540e8c1
      https://github.com/llvm/llvm-project/commit/460a2244430fae192298a5fd9fa2a269e540e8c1
  Author: Alexander Yermolovich <ayermolo at meta.com>
  Date:   2023-07-06 (Thu, 06 Jul 2023)

  Changed paths:
    A bolt/include/bolt/Core/DIEBuilder.h
    M bolt/include/bolt/Core/DebugData.h
    M bolt/include/bolt/Rewrite/DWARFRewriter.h
    M bolt/lib/Core/CMakeLists.txt
    A bolt/lib/Core/DIEBuilder.cpp
    M bolt/lib/Core/DebugData.cpp
    M bolt/lib/Rewrite/CMakeLists.txt
    M bolt/lib/Rewrite/DWARFRewriter.cpp
    M bolt/lib/Rewrite/RewriteInstance.cpp
    M bolt/test/X86/asm-func-debug.test
    M bolt/test/X86/dwarf3-lowpc-highpc-convert.s
    M bolt/test/X86/dwarf4-ftypes-dwp-input-dwo-output.test
    M bolt/test/X86/dwarf4-ftypes-dwp-input-dwp-output.test
    M bolt/test/X86/dwarf5-dwarf4-monolithic.test
    M bolt/test/X86/dwarf5-loclist-offset-form.test
    M bolt/test/X86/dwarf5-return-pc-form-addr.test
    M bolt/test/X86/dwarf5-split-dwarf4-monolithic.test
    M bolt/test/X86/shared-abbrev.s
    M llvm/include/llvm/CodeGen/DIE.h
    M llvm/include/llvm/DebugInfo/DWARF/DWARFFormValue.h
    M llvm/lib/DebugInfo/DWARF/DWARFFormValue.cpp

  Log Message:
  -----------
  [DWARF][BOLT] Implement new mechanism for DWARFRewriter

Summary:
This revision implement new mechanism for DWARFRewriter.
In the new mechanism, we adopt the same way with DWARFLinker did.
By parsing Debug information into IR, we are allowed to handle debug information more flexible.
Now the debug information updating process relies on IR and IR will be written out to binary once the updating finished.

A new class was added: DIEBuilder. This class is responsible for parsing debug information and raising it to the IR level.
This class is also used to write out the .debug_info and .debug_abbrev sections.
Since we output brand new Abbrev section we won't need to always convert low_pc/high_pc into ranges.
When conversion does happen we can also remove low_pc entry.

Differential Revision: https://phabricator.intern.facebook.com/D39484421

Tasks: T117448832


  Commit: 2701a661daa393ad5901ac88d420d7aa931eda0d
      https://github.com/llvm/llvm-project/commit/2701a661daa393ad5901ac88d420d7aa931eda0d
  Author: Alexander Yermolovich <ayermolo at meta.com>
  Date:   2023-07-06 (Thu, 06 Jul 2023)

  Changed paths:
    M bolt/include/bolt/Core/DIEBuilder.h
    M bolt/include/bolt/Core/DebugData.h
    M bolt/include/bolt/Rewrite/DWARFRewriter.h
    M bolt/lib/Core/DIEBuilder.cpp
    M bolt/lib/Core/DebugData.cpp
    M bolt/lib/Rewrite/DWARFRewriter.cpp
    A bolt/test/X86/Inputs/dwarf4-duplicate-types-helper.s
    A bolt/test/X86/Inputs/dwarf4-duplicate-types-main.s
    A bolt/test/X86/Inputs/dwarf4-gdb-index-types-helper.s
    A bolt/test/X86/Inputs/dwarf4-gdb-index-types-main.s
    A bolt/test/X86/Inputs/dwarf4-gdb-index-types-v8.gdb-index
    A bolt/test/X86/Inputs/dwarf4-split-gdb-index-types-helper.s
    A bolt/test/X86/Inputs/dwarf4-split-gdb-index-types-main.s
    A bolt/test/X86/Inputs/dwarf4-split-gdb-index-types-v8.gdb-index
    A bolt/test/X86/Inputs/dwarf4-types-dwarf5-helper.s
    A bolt/test/X86/Inputs/dwarf4-types-dwarf5-main.s
    A bolt/test/X86/Inputs/dwarf4-types-dwarf5-types-helper.s
    A bolt/test/X86/Inputs/dwarf4-types-dwarf5-types-main.s
    A bolt/test/X86/Inputs/dwarf4-types-helper.s
    A bolt/test/X86/Inputs/dwarf4-types-helper2.s
    A bolt/test/X86/Inputs/dwarf4-types-main.s
    A bolt/test/X86/Inputs/dwarf5-dwarf4-gdb-index-types-v8.generted-gdb11.gdb-index
    A bolt/test/X86/Inputs/dwarf5-dwarf4-gdb-index-types-v8.generted-gdb9.gdb-index
    A bolt/test/X86/Inputs/dwarf5-dwarf4-types-backward-forward-cross-reference-helper.s
    A bolt/test/X86/Inputs/dwarf5-dwarf4-types-backward-forward-cross-reference-main.s
    A bolt/test/X86/Inputs/dwarf5-gdb-index-types-helper.s
    A bolt/test/X86/Inputs/dwarf5-gdb-index-types-main.s
    A bolt/test/X86/Inputs/dwarf5-gdb-index-types-v8.generted-gdb11.gdb-index
    A bolt/test/X86/Inputs/dwarf5-gdb-index-types-v8.generted-gdb9.gdb-index
    A bolt/test/X86/Inputs/dwarf5-locexpr-referrence-helper.s
    A bolt/test/X86/Inputs/dwarf5-locexpr-referrence-main.s
    A bolt/test/X86/Inputs/dwarf5-split-gdb-index-types-helper.s
    A bolt/test/X86/Inputs/dwarf5-split-gdb-index-types-main.s
    A bolt/test/X86/Inputs/dwarf5-split-gdb-index-types-v8.gdb-index
    A bolt/test/X86/Inputs/dwarf5-types-helper2.s
    A bolt/test/X86/dwarf4-duplicate-types.test
    A bolt/test/X86/dwarf4-ftypes-dwo-mono-input-dwp-output.test
    A bolt/test/X86/dwarf4-gdb-index-types-gdb-generated.test
    A bolt/test/X86/dwarf4-gdb-index-types-lld-generated.test
    A bolt/test/X86/dwarf4-sibling.s
    A bolt/test/X86/dwarf4-split-gdb-index-types-gdb-generated.test
    A bolt/test/X86/dwarf4-types-dwarf5-types.test
    A bolt/test/X86/dwarf4-types-dwarf5.test
    A bolt/test/X86/dwarf4-types-forward-backward-cross-reference.s
    A bolt/test/X86/dwarf4-types.test
    A bolt/test/X86/dwarf5-dwarf4-gdb-index-types-gdb-generated-gdb11.test
    A bolt/test/X86/dwarf5-dwarf4-gdb-index-types-gdb-generated-gdb9.test
    A bolt/test/X86/dwarf5-dwarf4-gdb-index-types-lld-generated.test
    A bolt/test/X86/dwarf5-dwarf4-types-backward-forward-cross-reference.test
    A bolt/test/X86/dwarf5-ftypes-dwo-mono-input-dwp-output.test
    A bolt/test/X86/dwarf5-gdb-index-types-gdb-generated-gdb11.test
    A bolt/test/X86/dwarf5-gdb-index-types-gdb-generated-gdb9.test
    A bolt/test/X86/dwarf5-gdb-index-types-lld-generated.test
    A bolt/test/X86/dwarf5-locexpr-referrence.test
    A bolt/test/X86/dwarf5-split-gdb-index-types-gdb-generated.test
    A bolt/test/X86/dwarf5-types-backward-cross-reference.s
    A bolt/test/X86/dwarf5-types-forward-cross-reference.s

  Log Message:
  -----------
  [BOLT][DWARF] Numerous fixes for a new DWARFRewriter

Summary:

* Some cleanup and minor fixes for the new debug information re-writer before moving on
to productatization.

* The new rewriter wasn't handling binary with DWARF5 and DWARF4 with
-fdebug-types-sections.

* Removed dead cross cu reference code.

* Added support for DW_AT_sibling.

* With the new re-writer abbrev number can change which can lead to offset of Type
Units changing. Before we would just copy raw data. Changed to write out Type
Unit List. This is generated by gdb-add-index.

* Fixed how bolt handles gdb-index generated by gdb-11 with types sections.
Simplified logic that handles variations of gdb-index.

* Clang can generate two type units with the same hash, but different content. LLD
does not de-duplicate when ThinLTO is involved. Changed so that TU hash and
offset are used to make TU's unique.

* It is possible to have references within location expression to another DIE.
Fixed it so that relative offset is updated correctly.

* Removed all the code related to patching.

* Removed dead code. Changed how we handling writting out TUs and TU Index. It now
  should fully work for DWARF4 and DWARF5.

* Removed unused arguments from some APIs, changed return type to void, and other
small cleanups.

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: https://phabricator.intern.facebook.com/D46168257


  Commit: 46ca2e3fcd419b1246357ed3b9cd36630f16e64d
      https://github.com/llvm/llvm-project/commit/46ca2e3fcd419b1246357ed3b9cd36630f16e64d
  Author: Alexander Yermolovich <ayermolo at meta.com>
  Date:   2023-07-06 (Thu, 06 Jul 2023)

  Changed paths:
    M bolt/include/bolt/Rewrite/DWARFRewriter.h
    M bolt/lib/Rewrite/DWARFRewriter.cpp

  Log Message:
  -----------
  [BOLT][DWARF] Output DWO files as they are being processed

Summary:
Changed how we handle writing out .dwo and .dwp files. We now write out DWO
sections sooner and destroy DIEBuilder. This should decrease memory footprint.

Ran on clang-17 build in debug mode with split-dwarf.
before
8:07.49 real,   664.62 user,    69.00 sys,      0 amem, 41601612 mmem
8:07.06 real,   669.60 user,    68.75 sys,      0 amem, 41822588 mmem
8:00.36 real,   664.14 user,    66.36 sys,      0 amem, 41561548 mmem

after
8:21.85 real,   682.23 user,    69.64 sys,      0 amem, 39379880 mmem
8:04.58 real,   671.62 user,    66.50 sys,      0 amem, 39735800 mmem
8:10.02 real,   680.67 user,    67.24 sys,       0 amem, 39662888 mmem

Differential Revision: https://phabricator.intern.facebook.com/D45458889


  Commit: 88e95c1e4bb6e2ad3bfd185b96341ad5c09eff6b
      https://github.com/llvm/llvm-project/commit/88e95c1e4bb6e2ad3bfd185b96341ad5c09eff6b
  Author: Alexander Yermolovich <ayermolo at meta.com>
  Date:   2023-07-06 (Thu, 06 Jul 2023)

  Changed paths:
    M bolt/include/bolt/Core/DIEBuilder.h
    M bolt/include/bolt/Core/DebugData.h
    M bolt/include/bolt/Rewrite/DWARFRewriter.h
    M bolt/lib/Core/DIEBuilder.cpp
    M bolt/lib/Core/DebugData.cpp
    M bolt/lib/Rewrite/DWARFRewriter.cpp
    A bolt/test/X86/Inputs/dwarf4-cross-reference-different-abbrev-dst.s
    A bolt/test/X86/Inputs/dwarf4-cross-reference-different-abbrev-src.s
    M bolt/test/X86/Inputs/dwarf5-helper1-addr-section-reuse.s
    M bolt/test/X86/Inputs/dwarf5-helper2-addr-section-reuse.s
    A bolt/test/X86/dwarf4-cross-cu-backward-different-abbrev.test
    A bolt/test/X86/dwarf4-cross-cu-forward-different-abbrev.test
    M bolt/test/X86/dwarf4-types-dwarf5-types.test
    M bolt/test/X86/dwarf5-gdb-index-types-gdb-generated-gdb11.test
    M bolt/test/X86/dwarf5-gdb-index-types-gdb-generated-gdb9.test
    M bolt/test/X86/dwarf5-gdb-index-types-lld-generated.test
    M bolt/test/X86/dwarf5-locexpr-referrence.test

  Log Message:
  -----------
  [BOLT][DWARF] Change to process and write out TUs first then CUs in batches

Summary:
To reduce memory footprint changed so that we process and write out TUs first,
reset DIEBuilder and process CUs. CUs are processed in buckets. First bucket
contains all the CUs with cross CU references. Rest processd one at a time.

clang-17 build in debug mode, by clang-17.
before
8:25.81 real, 834.37 user, 86.03 sys, 0 amem, 79525064 mmem
8:02.20 real, 820.46 user, 81.81 sys, 0 amem, 79501616 mmem
7:52.69 real, 802.01 user, 83.99 sys, 0 amem, 79534392 mmem

after
7:49.35 real, 822.04 user, 66.19 sys, 0 amem, 34934260 mmem
7:42.16 real, 825.46 user, 63.52 sys, 0 amem, 34951660 mmem
7:46.71 real, 821.11 user, 63.14 sys, 0 amem, 34981164 mmem

Differential Revision: https://phabricator.intern.facebook.com/D45883198


Compare: https://github.com/llvm/llvm-project/compare/986001c8274a...88e95c1e4bb6

More information about the All-commits mailing list