[all-commits] [llvm/llvm-project] 7b8119: Introduce new symbol on-demand for debug info

[jeffreytan81 via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 7b81192d462bbd8031d5c665e29cd6b4c0c6887a
      https://github.com/llvm/llvm-project/commit/7b81192d462bbd8031d5c665e29cd6b4c0c6887a
  Author: Jeffrey Tan <jeffreytan at fb.com>
  Date:   2022-04-26 (Tue, 26 Apr 2022)

  Changed paths:
    A lldb/docs/use/ondemand.rst
    M lldb/include/lldb/Core/ModuleList.h
    M lldb/include/lldb/Symbol/SymbolFile.h
    A lldb/include/lldb/Symbol/SymbolFileOnDemand.h
    M lldb/include/lldb/Target/Statistics.h
    M lldb/include/lldb/Utility/LLDBLog.h
    M lldb/source/Core/CoreProperties.td
    M lldb/source/Core/Module.cpp
    M lldb/source/Core/ModuleList.cpp
    M lldb/source/Plugins/SymbolFile/DWARF/DWARFIndex.cpp
    M lldb/source/Plugins/SymbolFile/DWARF/DebugNamesDWARFIndex.cpp
    M lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp
    M lldb/source/Plugins/SymbolFile/PDB/PDBASTParser.cpp
    M lldb/source/Symbol/CMakeLists.txt
    M lldb/source/Symbol/CompileUnit.cpp
    M lldb/source/Symbol/SymbolFile.cpp
    A lldb/source/Symbol/SymbolFileOnDemand.cpp
    M lldb/source/Target/Statistics.cpp
    M lldb/source/Utility/LLDBLog.cpp
    A lldb/test/API/symbol_ondemand/breakpoint_language/Makefile
    A lldb/test/API/symbol_ondemand/breakpoint_language/TestBreakpointLanguageOnDemand.py
    A lldb/test/API/symbol_ondemand/breakpoint_language/c_lang.c
    A lldb/test/API/symbol_ondemand/breakpoint_language/cpp_lang.cpp
    A lldb/test/API/symbol_ondemand/breakpoint_language/main.cpp
    A lldb/test/API/symbol_ondemand/breakpoint_source_regex/Makefile
    A lldb/test/API/symbol_ondemand/breakpoint_source_regex/TestSourceTextRegexBreakpoint.py
    A lldb/test/API/symbol_ondemand/breakpoint_source_regex/main.cpp
    A lldb/test/API/symbol_ondemand/shared_library/Makefile
    A lldb/test/API/symbol_ondemand/shared_library/TestSharedLib.py
    A lldb/test/API/symbol_ondemand/shared_library/foo.c
    A lldb/test/API/symbol_ondemand/shared_library/foo.h
    A lldb/test/API/symbol_ondemand/shared_library/shared.c
    A lldb/test/Shell/SymbolFile/OnDemand/Inputs/basic.cpp
    A lldb/test/Shell/SymbolFile/OnDemand/source-breakpoint.test
    A lldb/test/Shell/SymbolFile/OnDemand/symbolic-breakpoint.test

  Log Message:
  -----------
  Introduce new symbol on-demand for debug info

This diff introduces a new symbol on-demand which skips
loading a module's debug info unless explicitly asked on
demand. This provides significant performance improvement
for application with dynamic linking mode which has large
number of modules.
The feature can be turned on with:
"settings set symbols.load-on-demand true"

The feature works by creating a new SymbolFileOnDemand class for
each module which wraps the actual SymbolFIle subclass as member
variable. By default, most virtual methods on SymbolFileOnDemand are
skipped so that it looks like there is no debug info for that module.
But once the module's debug info is explicitly requested to
be enabled (in the conditions mentioned below) SymbolFileOnDemand
will allow all methods to pass through and forward to the actual SymbolFile
which would hydrate module's debug info on-demand.

In an internal benchmark, we are seeing more than 95% improvement
for a 3000 modules application.

Currently we are providing several ways to on demand hydrate
a module's debug info:
* Source line breakpoint: matching in supported files
* Stack trace: resolving symbol context for an address
* Symbolic breakpoint: symbol table match guided promotion
* Global variable: symbol table match guided promotion

In all above situations the module's debug info will be on-demand
parsed and indexed.

Some follow-ups for this feature:
* Add a command that allows users to load debug info explicitly while using a
  new or existing command when this feature is enabled
* Add settings for "never load any of these executables in Symbols On Demand"
  that takes a list of globs
* Add settings for "always load the the debug info for executables in Symbols
  On Demand" that takes a list of globs
* Add a new column in "image list" that shows up by default when Symbols On
  Demand is enable to show the status for each shlib like "not enabled for
  this", "debug info off" and "debug info on" (with a single character to
  short string, not the ones I just typed)

Differential Revision: https://reviews.llvm.org/D121631