[Lldb-commits] [lldb] f7a49d2 - [WIP][DebugInfo] Lazily parse debug_loclist offsets

David Blaikie via lldb-commits lldb-commits at lists.llvm.org
Tue Aug 18 10:50:05 PDT 2020


Author: David Blaikie
Date: 2020-08-18T10:49:39-07:00
New Revision: f7a49d2aa691266497c4baa35f29ba0167b39d23

URL: https://github.com/llvm/llvm-project/commit/f7a49d2aa691266497c4baa35f29ba0167b39d23
DIFF: https://github.com/llvm/llvm-project/commit/f7a49d2aa691266497c4baa35f29ba0167b39d23.diff

LOG: [WIP][DebugInfo] Lazily parse debug_loclist offsets

Parsing DWARFv5 debug_loclist offsets when a CU is parsed is weighing
down memory usage of symbolizers that don't need to parse this data at
all. There's not much benefit to caching these anyway - since they are
O(1) lookup and reading once you know where the offset list starts (and
can do bounds checking with the offset list size too).

In general, I think it might be time to start paying down some of the
technical debt of loc/loclist/range/rnglist parsing to try to unify it a
bit more.

eg:

* Currently DWARFUnit has: RangeSection, RangeSectionBase, LocSection,
  LocSectionBase, LocTable, RngListTable, LoclistTableHeader (be nice if
  these were all wrapped up in two variables - one for loclists, one for
  rnglists)

* rnglists and loclists are handled differently (see:
  LoclistTableHeader, but no RnglistTableHeader)

* maybe all these types could be less stateful - lazily parse what they
  need to, even reparsing rather than caching because it doesn't seem
  too expensive, for instance. (though admittedly so long as it's
  constantcost/overead per compilatiton that's probably adequate)

* Maybe implementing and using a DWARFDataExtractor that can be
  sub-ranged (so we could slice it up to just the single contribution) -
  though maybe that's not so useful because loc/ranges need to refer to
  it by absolute, not contribution-relative mechanisms

Differential Revision: https://reviews.llvm.org/D86110

Added: 
    

Modified: 
    lldb/source/Plugins/SymbolFile/DWARF/DWARFUnit.h
    lldb/test/Shell/SymbolFile/DWARF/DW_AT_loclists_base.s
    llvm/include/llvm/DebugInfo/DWARF/DWARFDebugLoc.h
    llvm/include/llvm/DebugInfo/DWARF/DWARFListTable.h
    llvm/include/llvm/DebugInfo/DWARF/DWARFUnit.h
    llvm/lib/DebugInfo/DWARF/DWARFContext.cpp
    llvm/lib/DebugInfo/DWARF/DWARFListTable.cpp
    llvm/lib/DebugInfo/DWARF/DWARFUnit.cpp

Removed: 
    


################################################################################
diff  --git a/lldb/source/Plugins/SymbolFile/DWARF/DWARFUnit.h b/lldb/source/Plugins/SymbolFile/DWARF/DWARFUnit.h
index affad286a490..1d8236c4ed42 100644
--- a/lldb/source/Plugins/SymbolFile/DWARF/DWARFUnit.h
+++ b/lldb/source/Plugins/SymbolFile/DWARF/DWARFUnit.h
@@ -237,7 +237,9 @@ class DWARFUnit : public lldb_private::UserID {
   llvm::Optional<uint64_t> GetRnglistOffset(uint32_t Index) const {
     if (!m_rnglist_table)
       return llvm::None;
-    if (llvm::Optional<uint64_t> off = m_rnglist_table->getOffsetEntry(Index))
+    if (llvm::Optional<uint64_t> off = m_rnglist_table->getOffsetEntry(
+            m_dwarf.GetDWARFContext().getOrLoadRngListsData().GetAsLLVM(),
+            Index))
       return *off + m_ranges_base;
     return llvm::None;
   }
@@ -246,7 +248,8 @@ class DWARFUnit : public lldb_private::UserID {
     if (!m_loclist_table_header)
       return llvm::None;
 
-    llvm::Optional<uint64_t> Offset =  m_loclist_table_header->getOffsetEntry(Index);
+    llvm::Optional<uint64_t> Offset = m_loclist_table_header->getOffsetEntry(
+        m_dwarf.GetDWARFContext().getOrLoadLocListsData().GetAsLLVM(), Index);
     if (!Offset)
       return llvm::None;
     return *Offset + m_loclists_base;

diff  --git a/lldb/test/Shell/SymbolFile/DWARF/DW_AT_loclists_base.s b/lldb/test/Shell/SymbolFile/DWARF/DW_AT_loclists_base.s
index ca32e9930a76..6ff35f8c6596 100644
--- a/lldb/test/Shell/SymbolFile/DWARF/DW_AT_loclists_base.s
+++ b/lldb/test/Shell/SymbolFile/DWARF/DW_AT_loclists_base.s
@@ -5,7 +5,7 @@
 
 # CHECK-LABEL: image lookup -v -s lookup_loclists
 # CHECK: Variable: {{.*}}, name = "x0", type = "int", location = DW_OP_reg0 RAX,
-# CHECK: Variable: {{.*}}, name = "x1", type = "int", location = ,
+# CHECK-NOT: Variable:
 
 loclists:
         nop
@@ -28,7 +28,7 @@ lookup_loclists:
         .short  5                       # Version
         .byte   8                       # Address size
         .byte   0                       # Segment selector size
-        .long   1                       # Offset entry count
+        .long   2                       # Offset entry count
 .Lloclists_table_base:
         .long   .Ldebug_loc0-.Lloclists_table_base
         .long   .Ldebug_loc1-.Lloclists_table_base

diff  --git a/llvm/include/llvm/DebugInfo/DWARF/DWARFDebugLoc.h b/llvm/include/llvm/DebugInfo/DWARF/DWARFDebugLoc.h
index 3b141304f85f..dbc11c51a789 100644
--- a/llvm/include/llvm/DebugInfo/DWARF/DWARFDebugLoc.h
+++ b/llvm/include/llvm/DebugInfo/DWARF/DWARFDebugLoc.h
@@ -72,6 +72,8 @@ class DWARFLocationTable {
       std::function<Optional<object::SectionedAddress>(uint32_t)> LookupAddr,
       function_ref<bool(Expected<DWARFLocationExpression>)> Callback) const;
 
+  const DWARFDataExtractor &getData() { return Data; }
+
 protected:
   DWARFDataExtractor Data;
 

diff  --git a/llvm/include/llvm/DebugInfo/DWARF/DWARFListTable.h b/llvm/include/llvm/DebugInfo/DWARF/DWARFListTable.h
index 496fdb2477f9..bcfc71381aee 100644
--- a/llvm/include/llvm/DebugInfo/DWARF/DWARFListTable.h
+++ b/llvm/include/llvm/DebugInfo/DWARF/DWARFListTable.h
@@ -72,10 +72,6 @@ class DWARFListTableHeader {
   };
 
   Header HeaderData;
-  /// The offset table, which contains offsets to the individual list entries.
-  /// It is used by forms such as DW_FORM_rnglistx.
-  /// FIXME: Generate the table and use the appropriate forms.
-  std::vector<uint64_t> Offsets;
   /// The table's format, either DWARF32 or DWARF64.
   dwarf::DwarfFormat Format;
   /// The offset at which the header (and hence the table) is located within
@@ -93,7 +89,6 @@ class DWARFListTableHeader {
 
   void clear() {
     HeaderData = {};
-    Offsets.clear();
   }
   uint64_t getHeaderOffset() const { return HeaderOffset; }
   uint8_t getAddrSize() const { return HeaderData.AddrSize; }
@@ -115,11 +110,17 @@ class DWARFListTableHeader {
     llvm_unreachable("Invalid DWARF format (expected DWARF32 or DWARF64");
   }
 
-  void dump(raw_ostream &OS, DIDumpOptions DumpOpts = {}) const;
-  Optional<uint64_t> getOffsetEntry(uint32_t Index) const {
-    if (Index < Offsets.size())
-      return Offsets[Index];
-    return None;
+  void dump(DataExtractor Data, raw_ostream &OS,
+            DIDumpOptions DumpOpts = {}) const;
+  Optional<uint64_t> getOffsetEntry(DataExtractor Data, uint32_t Index) const {
+    if (Index > HeaderData.OffsetEntryCount)
+      return None;
+
+    uint8_t OffsetByteSize = Format == dwarf::DWARF64 ? 8 : 4;
+    uint64_t Offset =
+        getHeaderOffset() + getHeaderSize(Format) + OffsetByteSize * Index;
+    auto R = Data.getUnsigned(&Offset, OffsetByteSize);
+    return R;
   }
 
   /// Extract the table header and the array of offsets.
@@ -169,14 +170,14 @@ template <typename DWARFListType> class DWARFListTableBase {
   uint8_t getAddrSize() const { return Header.getAddrSize(); }
   dwarf::DwarfFormat getFormat() const { return Header.getFormat(); }
 
-  void dump(raw_ostream &OS,
+  void dump(DWARFDataExtractor Data, raw_ostream &OS,
             llvm::function_ref<Optional<object::SectionedAddress>(uint32_t)>
                 LookupPooledAddress,
             DIDumpOptions DumpOpts = {}) const;
 
   /// Return the contents of the offset entry designated by a given index.
-  Optional<uint64_t> getOffsetEntry(uint32_t Index) const {
-    return Header.getOffsetEntry(Index);
+  Optional<uint64_t> getOffsetEntry(DataExtractor Data, uint32_t Index) const {
+    return Header.getOffsetEntry(Data, Index);
   }
   /// Return the size of the table header including the length but not including
   /// the offsets. This is dependent on the table format, which is unambiguously
@@ -240,11 +241,11 @@ Error DWARFListType<ListEntryType>::extract(DWARFDataExtractor Data,
 
 template <typename DWARFListType>
 void DWARFListTableBase<DWARFListType>::dump(
-    raw_ostream &OS,
+    DWARFDataExtractor Data, raw_ostream &OS,
     llvm::function_ref<Optional<object::SectionedAddress>(uint32_t)>
         LookupPooledAddress,
     DIDumpOptions DumpOpts) const {
-  Header.dump(OS, DumpOpts);
+  Header.dump(Data, OS, DumpOpts);
   OS << HeaderString << "\n";
 
   // Determine the length of the longest encoding string we have in the table,

diff  --git a/llvm/include/llvm/DebugInfo/DWARF/DWARFUnit.h b/llvm/include/llvm/DebugInfo/DWARF/DWARFUnit.h
index 1613e74d4a36..c76ee5efa37b 100644
--- a/llvm/include/llvm/DebugInfo/DWARF/DWARFUnit.h
+++ b/llvm/include/llvm/DebugInfo/DWARF/DWARFUnit.h
@@ -412,18 +412,13 @@ class DWARFUnit {
   /// Return a rangelist's offset based on an index. The index designates
   /// an entry in the rangelist table's offset array and is supplied by
   /// DW_FORM_rnglistx.
-  Optional<uint64_t> getRnglistOffset(uint32_t Index) {
-    if (!RngListTable)
-      return None;
-    if (Optional<uint64_t> Off = RngListTable->getOffsetEntry(Index))
-      return *Off + RangeSectionBase;
-    return None;
-  }
+  Optional<uint64_t> getRnglistOffset(uint32_t Index);
 
   Optional<uint64_t> getLoclistOffset(uint32_t Index) {
     if (!LoclistTableHeader)
       return None;
-    if (Optional<uint64_t> Off = LoclistTableHeader->getOffsetEntry(Index))
+    if (Optional<uint64_t> Off =
+            LoclistTableHeader->getOffsetEntry(LocTable->getData(), Index))
       return *Off + getLocSectionBase();
     return None;
   }

diff  --git a/llvm/lib/DebugInfo/DWARF/DWARFContext.cpp b/llvm/lib/DebugInfo/DWARF/DWARFContext.cpp
index 3bcde8fafb1f..96ba57946833 100644
--- a/llvm/lib/DebugInfo/DWARF/DWARFContext.cpp
+++ b/llvm/lib/DebugInfo/DWARF/DWARFContext.cpp
@@ -255,7 +255,7 @@ static void dumpRnglistsSection(
         break;
       Offset = TableOffset + Length;
     } else {
-      Rnglists.dump(OS, LookupPooledAddress, DumpOpts);
+      Rnglists.dump(rnglistData, OS, LookupPooledAddress, DumpOpts);
     }
   }
 }
@@ -316,7 +316,7 @@ static void dumpLoclistsSection(raw_ostream &OS, DIDumpOptions DumpOpts,
       return;
     }
 
-    Header.dump(OS, DumpOpts);
+    Header.dump(Data, OS, DumpOpts);
 
     uint64_t EndOffset = Header.length() + Header.getHeaderOffset();
     Data.setAddressSize(Header.getAddrSize());

diff  --git a/llvm/lib/DebugInfo/DWARF/DWARFListTable.cpp b/llvm/lib/DebugInfo/DWARF/DWARFListTable.cpp
index 2124a49bef60..c876af1e9b51 100644
--- a/llvm/lib/DebugInfo/DWARF/DWARFListTable.cpp
+++ b/llvm/lib/DebugInfo/DWARF/DWARFListTable.cpp
@@ -71,12 +71,12 @@ Error DWARFListTableHeader::extract(DWARFDataExtractor Data,
         ") than there is space for",
         SectionName.data(), HeaderOffset, HeaderData.OffsetEntryCount);
   Data.setAddressSize(HeaderData.AddrSize);
-  for (uint32_t I = 0; I < HeaderData.OffsetEntryCount; ++I)
-    Offsets.push_back(Data.getRelocatedValue(OffsetByteSize, OffsetPtr));
+  *OffsetPtr += HeaderData.OffsetEntryCount * OffsetByteSize;
   return Error::success();
 }
 
-void DWARFListTableHeader::dump(raw_ostream &OS, DIDumpOptions DumpOpts) const {
+void DWARFListTableHeader::dump(DataExtractor Data, raw_ostream &OS,
+                                DIDumpOptions DumpOpts) const {
   if (DumpOpts.Verbose)
     OS << format("0x%8.8" PRIx64 ": ", HeaderOffset);
   int OffsetDumpWidth = 2 * dwarf::getDwarfOffsetByteSize(Format);
@@ -91,7 +91,8 @@ void DWARFListTableHeader::dump(raw_ostream &OS, DIDumpOptions DumpOpts) const {
 
   if (HeaderData.OffsetEntryCount > 0) {
     OS << "offsets: [";
-    for (const auto &Off : Offsets) {
+    for (uint32_t I = 0; I < HeaderData.OffsetEntryCount; ++I) {
+      auto Off = *getOffsetEntry(Data, I);
       OS << format("\n0x%0*" PRIx64, OffsetDumpWidth, Off);
       if (DumpOpts.Verbose)
         OS << format(" => 0x%08" PRIx64,

diff  --git a/llvm/lib/DebugInfo/DWARF/DWARFUnit.cpp b/llvm/lib/DebugInfo/DWARF/DWARFUnit.cpp
index 0527f29d1a1a..b871e6ebdca5 100644
--- a/llvm/lib/DebugInfo/DWARF/DWARFUnit.cpp
+++ b/llvm/lib/DebugInfo/DWARF/DWARFUnit.cpp
@@ -548,17 +548,13 @@ Error DWARFUnit::tryExtractDIEsIfNeeded(bool CUDieOnly) {
 
       uint64_t HeaderSize = DWARFListTableHeader::getHeaderSize(Header.getFormat());
       uint64_t Offset = getLocSectionBase();
-      DWARFDataExtractor Data(Context.getDWARFObj(), *LocSection,
-                              isLittleEndian, getAddressByteSize());
+      const DWARFDataExtractor &Data = LocTable->getData();
       if (Offset < HeaderSize)
         return createStringError(errc::invalid_argument,
                                  "did not detect a valid"
                                  " list table with base = 0x%" PRIx64 "\n",
                                  Offset);
       Offset -= HeaderSize;
-      if (auto *IndexEntry = Header.getIndexEntry())
-        if (const auto *Contrib = IndexEntry->getContribution(DW_SECT_LOCLISTS))
-          Offset += Contrib->Offset;
       if (Error E = LoclistTableHeader->extract(Data, &Offset))
         return createStringError(errc::invalid_argument,
                                  "parsing a loclist table: " +
@@ -1009,3 +1005,13 @@ DWARFUnit::determineStringOffsetsTableContributionDWO(DWARFDataExtractor & DA) {
     return DescOrError.takeError();
   return *DescOrError;
 }
+
+Optional<uint64_t> DWARFUnit::getRnglistOffset(uint32_t Index) {
+  if (!RngListTable)
+    return None;
+  DataExtractor RangesData(RangeSection->Data, isLittleEndian,
+                           getAddressByteSize());
+  if (Optional<uint64_t> Off = RngListTable->getOffsetEntry(RangesData, Index))
+    return *Off + RangeSectionBase;
+  return None;
+}


        


More information about the lldb-commits mailing list