<div dir="ltr">I think making debug info more of a first-class IR citizen is probably the way to go. Right now debug info is completely unreadable and is downright opposed to the design goals of the IR as I understand them.<div><br></div><div>Our backwards compatibility policy should give you the flexibility you need to update the debug info representation as you go along: <a href="http://llvm.org/docs/DeveloperPolicy.html#id18">http://llvm.org/docs/DeveloperPolicy.html#id18</a></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Oct 13, 2014 at 3:02 PM, Duncan P. N. Exon Smith <span dir="ltr"><<a href="mailto:dexonsmith@apple.com" target="_blank">dexonsmith@apple.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">In r219010, I merged integer and string fields into a single header<br>
field. By reducing the number of metadata operands used in debug info,<br>
this saved 2.2GB on an `llvm-lto` bootstrap. I've done some profiling<br>
of DW_TAGs to see what parts of PR17891 and PR17892 to tackle next, and<br>
I've concluded that they will be insufficient.<br>
<br>
Instead, I'd like to implement a more aggressive plan, which as a<br>
side-effect cleans up the much "loved" debug info IR assembly syntax.<br>
<br>
At a high-level, the idea is to create distinct subclasses of `Value`<br>
for each debug info concept, starting with line table entries and moving<br>
on to the DIDescriptor hierarchy. By leveraging the use-list<br>
infrastructure for metadata operands -- i.e., only using value handles<br>
for non-metadata operands -- we'll improve memory usage and increase<br>
RAUW speed.<br>
<br>
My rough plan follows. I quote some numbers for memory savings below<br>
based on an -flto -g bootstrap of `llvm-lto` (i.e., running `llvm-lto`<br>
on `llvm-lto.lto.bc`, an already-linked bitcode file dumped by ld64's<br>
-save-temps option) that currently peaks at 15.3GB.<br>
<br>
1. Introduce `MDUser`, which inherits from `User`, and whose `Use`s<br>
must all be metadata. The cost per operand is 1 pointer, vs. 4<br>
pointers in an `MDNode`.<br>
<br>
2. Create `MDLineTable` as the first subclass of `MDUser`. Use normal<br>
fields (not `Value`s) for the line and column, and use `Use`<br>
operands for the metadata operands.<br>
<br>
On x86-64, this will save 104B / line table entry. Linking<br>
`llvm-lto` uses ~7M line-table entries, so this on its own saves<br>
~700MB.<br>
<br>
Sketch of class definition:<br>
<br>
class MDLineTable : public MDUser {<br>
unsigned Line;<br>
unsigned Column;<br>
public:<br>
static MDLineTable *get(unsigned Line, unsigned Column,<br>
MDNode *Scope);<br>
static MDLineTable *getInlined(MDLineTable *Base, MDNode *Scope);<br>
static MDLineTable *getBase(MDLineTable *Inlined);<br>
<br>
unsigned getLine() const { return Line; }<br>
unsigned getColumn() const { return Column; }<br>
bool isInlined() const { return getNumOperands() == 2; }<br>
MDNode *getScope() const { return getOperand(0); }<br>
MDNode *getInlinedAt() const { return getOperand(1); }<br>
};<br>
<br>
Proposed assembly syntax:<br>
<br>
; Not inlined.<br>
!7 = metadata !MDLineTable(line: 45, column: 7, scope: metadata !9)<br>
<br>
; Inlined.<br>
!7 = metadata !MDLineTable(line: 45, column: 7, scope: metadata !9,<br>
inlinedAt: metadata !10)<br>
<br>
; Column defaulted to 0.<br>
!7 = metadata !MDLineTable(line: 45, scope: metadata !9)<br>
<br>
(What colour should that bike shed be?)<br>
<br>
3. (Optional) Rewrite `DebugLoc` lookup tables. My profiling shows<br>
that we have 3.5M entries in the `DebugLoc` side-vectors for 7M line<br>
table entries. The cost of these is ~180B each, for another<br>
~600MB.<br>
<br>
If we integrate a side-table of `MDLineTable`s into its uniquing,<br>
the overhead is only ~12B / line table entry, or ~80MB. This saves<br>
520MB.<br>
<br>
This is somewhat perpendicular to redesigning the metadata format,<br>
but IMO it's worth doing as soon as it's possible.<br>
<br>
4. Create `GenericDebugMDNode`, a transitional subclass of `MDUser`<br>
through an intermediate class `DebugMDNode` with an<br>
allocation-time-optional `CallbackVH` available for referencing<br>
non-metadata. Change `DIDescriptor` to wrap a `DebugMDNode` instead<br>
of an `MDNode`.<br>
<br>
This saves another ~960MB, for a running total of ~2GB.<br>
<br>
Proposed assembly syntax:<br>
<br>
!7 = metadata !GenericDebugMDNode(tag: DW_TAG_compile_unit,<br>
fields: "0\00clang 3.6\00...",<br>
operands: { metadata !8, ... })<br>
<br>
!7 = metadata !GenericDebugMDNode(tag: DW_TAG_variable,<br>
fields: "global_var\00...",<br>
operands: { metadata !8, ... },<br>
handle: i32* @global_var)<br>
<br>
This syntax pulls the tag out of the current header-string, calls<br>
the rest of the header "fields", and includes the metadata operands<br>
in "operands".<br>
<br>
5. Incrementally create subclasses of `DebugMDNode`, such as<br>
`MDCompileUnit` and `MDSubprogram`. Sub-classed nodes replace the<br>
"fields" and "operands" catch-alls with explicit names for each<br>
operand.<br>
<br>
Proposed assembly syntax:<br>
<br>
!7 = metadata !MDSubprogram(line: 45, name: "foo", displayName: "foo",<br>
linkageName: "_Z3foov", file: metadata !8,<br>
function: i32 (i32)* @foo)<br>
<br>
6. Remove the dead code for `GenericDebugMDNode`.<br>
<br>
7. (Optional) Refactor `DebugMDNode` sub-classes to minimize RAUW<br>
traffic during bitcode serialization. Now that metadata types are<br>
known, we can write debug info out in an order that makes it cheap<br>
to read back in.<br>
<br>
Note that using `MDUser` will make RAUW much cheaper, since we're<br>
using the use-list infrastructure for most of them. If RAUW isn't<br>
showing up in a profile, I may skip this.<br>
<br>
Does this direction seem reasonable? Any major problems I've missed?<br>
<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
</blockquote></div><br></div>