<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>LLVM Debug Info with Metadata</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="author" content="Renato Golin">
<meta name="description"
content="LLVM Debug Info with Metadata">
<link rel="stylesheet" href="http://llvm.org/docs/llvm.css" type="text/css">
</head>
<body>
<div class="doc_title"> LLVM Debug Info with Metadata</div>
<ol>
<li><a href="#abstract">Abstract</a></li>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#constructing">Constructing Debug Information</a></li>
<ol>
<li><a href="#factory">DIFactory</a></li>
<li><a href="#compileunit">DICompileUnit</a></li>
<li><a href="#file">DIFile</a></li>
<li><a href="#type">DIType</a></li>
<li><a href="#global">DIGlobalVariable</a></li>
<li><a href="#subprogram">DISubprogram</a></li>
<li><a href="#variable">DIVariable</a></li>
</ol>
<li><a href="#accessing">Accessing Debug Information</a></li>
<li><a href="#line_info">Debug Line Information</a></li>
<li><a href="#path">Debug Line Path within LLVM</a></li>
<ol>
<li><a href="#path-fe">Front-end</a></li>
<li><a href="#path-be">Back-end</a></li>
</ol>
<li><a href="#discussions">Further Discussions</a></li>
</ol>
<div class="doc_author">
<p>Written by <a href="mailto:rengolin@systemcall.org">Renato Golin</a></p>
</div>
<!-- *********************************************************************** -->
<div class="doc_section"> <a name="abstract">Abstract</a></div>
<!-- *********************************************************************** -->
<div class="doc_text">
<p>How to produce debug information in LLVM has changed considerably since 2.7. A new
<a href="http://blog.llvm.org/2010/04/extensible-metadata-in-llvm-ir.html">extensible metadata</a>
engine provides a flexible way of attaching extra information to LLVM Values and to make sure
only needed additional information will be kept in the IR. There is already a long document on how the debug information is
<a href="http://llvm.org/docs/SourceLevelDebugging.html">presented in the IR</a>, but here, we provide a simple description
on how to use the DebugInfo API for building debug information in the IR level from a generic AST descriptions.</p>
</div>
<!-- *********************************************************************** -->
<div class="doc_section"> <a name="introduction">Introduction</a></div>
<!-- *********************************************************************** -->
<div class="doc_text">
<p>This document describes how to use the DI helpers to build debug information, starting with an AST generated by a typical frontend.
LLVM Values do not capture all of the information in the AST, such as the original declared types and qualifiers, or position information
such as file/line/column. For this reason, DI helpers are used to annotate the LLVM Values so that this information can be recovered later.
</p>
<p>The basic idea about building debug information is that of using metadata to represent the DWARF fields of each Value. In LLVM,
metadata is represented by a <a href="http://llvm.org/docs/LangRef.html#metadata">MDNode</a>, which points to a Value. MDNodes can
have any number of fields, so you can store different DWARF information (line info, type info, etc) using only MDNodes.</p>
<p>In order to make sure the MDNodes are built correctly (with the correct number of parameters in the right order), some helper classes
were written to help you extract information from (and put a type on) MDNodes. Those are the DI* classes, all deriving from DIDescriptor.</p>
<p>Also, MDNodes are only exported if they point to some Value (including other MDNodes) that are also exported. This is a simple and
efficient way of pruning unused metadata.</p>
<p>In the IR level, only the metadata is present, no DWARF information is generated. Later on, in the CodeGen phase, the debug backend
take this metadata and transfrom it into DWARF symbols, either by the ASM printer or the object writer in the MC infrastructure.</p>
</div>
<!-- *********************************************************************** -->
<div class="doc_section"> <a name="constructing">Constructing Debug Information</a></div>
<!-- *********************************************************************** -->
<div class="doc_text">
<p>All DI Helpers are composed of an MDNode pointer and a set of utility member functions to extract information from the node.</p>
<p>Each DI* class can be constructed using the DIFactory, which accepts a list of MDNodes and assign them to an internal array
in the right order. There is no veryfication of types in this building process yet (see <a href="#discussions">discussions</a> below),
so please make sure you're passing the correct parameters in the correct order.</p>
<p>The basic hierarchy for a normal program is to have <strong>compilation units</strong> as the base of the debug tree, with
<strong>files</strong>, <strong>sub-programs</strong> (functions, procedures) and <strong>global variables</strong> attached to them.
Each <strong>variable</strong> (global or local) has a <strong>type</strong> associated.</p>
<p>Files are NOT compilation units. In languages like C, the compiled file (say <em>foo.c</em> in <code>clang -c foo.c</code>) is both
a <strong>file</strong> and a <strong>compilation units</strong>. Included files, like <code>stdio.h</code> are only <strong>files</strong>.</p>
<p>All statements belong to a <strong>lexical block</strong>, that itself can belong to another, in a tree structure. The root
<strong>lexical block</strong> in each tree belongs to a <strong>sub-program</strong>.</p>
<h5>Graphical example:</h5>
<p>Take the following C program as an example (<code>foo.c</code>):</p>
<pre>
1: #include <stdio.h>
2:
3: int global = 10;
4:
5: int main() {
6: int local = 5;
7: for (; local < global; local++)
8: printf("Local = %d", local);
9: }
</pre>
<p>Could be represented as:</p>
<ul>
<LI>CompileUnit: foo.c</LI>
<ul>
<LI>File: foo.c</LI>
<LI>File: stdio.h</LI>
<LI>GlobalVar: global [line 3, col 5]</LI>
<ul>
<LI>Type: int</LI>
</ul>
<LI>SubProgram: main [line 5, col 5]</LI>
<ul>
<LI>Type: function (int (void))</LI>
<LI>LexicalBlock (whole main) [line 5, col 12]</LI>
<ul>
<LI>Variable: local [line 6, col 9]</LI>
<ul>
<LI>Type: int</LI>
</ul>
<LI>LexicalBlock (for increment) [line 7, col 26]</LI>
<LI>LexicalBlock (for body) [line 8, col 8]</LI>
</ul>
</ul>
</ul>
</ul>
<p>However incomplete this representation is, it gives an idea on how the basic elements are stored to provide a meaningful
Dwarf information for debuggers to be able to step throug your program and inspect variables and expressions.</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection">
<a name="factory"><a href="http://llvm.org/doxygen/classllvm_1_1DIFactory.html">DIFactory</a></a>
</div>
<div class="doc_text">
<p>Contains all methods for creating DI* objects (and its MDNodes) from the list of parameters passed to the functions.</p>
<p>Use these methods to create all debug information (rather than creating the arrays manually). It provides some basic
guarantees on how many arguments and the basic type (DI* type) of some of them. While they're not actually validating
everything yet, it's the logical place to add validation in the future.</p>
<h5>Basic Helper Functions:</h5>
<pre class="doc_code">// Creates a compilation unit
DICompileUnit CreateCompileUnit(...);
// Creates a file associated to that compilation unit
DIFile CreateFile(..., DICompileUnit CU);
// Creates a function inside that file
DISubprogram CreateSubprogram(..., DIFile F, ...);
// Creates a lexical block inside that function (to which variables will be associated with)
DILexicalBlock CreateLexicalBlock(DIDescriptor Context, ...);
// Creates a variable inside a lexical block
DIVariable CreateVariable(..., DIDescriptor Context, ...);
// Creates the type of that variable, function
DIBasicType CreateBasicType(DIDescriptor Context, ...);
</pre>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection">
<a name="compileunit"><a href="http://llvm.org/doxygen/classllvm_1_1DICompileUnit.html">DICompileUnit</a></a>
</div>
<div class="doc_text">
<p>Compilation unit is the file you are compiling, generally passed vua command line. So, when you say <code>compiler file.c</code>,
<code>file.c</code> is your compilation unit. As you'll see below, it is also a file, along with its included files, that get attached
to this compilation unit.</p>
<p>When compiling multiple files in the same command (ex. <code>compiler file1.c file2.c</code>), each file is a separate compilation
unit.</p>
<p>To create a compilation unit, we need a few information about the file and the system:</p>
<ol>
<li>The <b>language</b> used on the file, from the <code>Dwarf.h</code> header file, for example <code>dwarf::DW_LANG_C99</code>.</li>
<li>The <b>file name</b>.</li>
<li>The <b>directory</b> containing the file. This could be the current directory <code>.</code> or the full path.</li>
<li>The <b>compiler</b> name, version and whatever you feel important to put in that identifies your compiler.</li>
</ol>
<h5>Example:</h5>
<pre class="doc_code">DICompileUnit cunit = factory.CreateCompileUnit(
dwarf::DW_LANG_C99,
"main.c",
"/home/username/src",
"my pet compiler v1.0");
</pre>
<p>Save that object (context, global), as you'll need it for the main file itself and all additional files that were included from it.</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection">
<a name="file"><a href="http://llvm.org/doxygen/classllvm_1_1DIFile.html">DIFile</a></a>
</div>
<div class="doc_text">
<p>Files are all source and header files included in a compilation unit. Line information must be associated with a specific file, so it's
important that you keep track of the files correctly and on which line/column the statement appears.</p>
<p>For every file, source or header, you need to create a DIFile for it. With DIFiles, you can attach functions (DISubprogram) and global
variables (DIGlobalVariable). The parameters are:</p>
<ol>
<li>The <b>file name</b>.</li>
<li>The <b>directory</b> containing the file.</li>
<li>The <b>compilation unit</b> created above.</li>
</ol>
<h5>Example:</h5>
<pre class="doc_code">DIFile file = factory.CreateFile(
"main.c",
"/home/username/",
cunit);
</pre>
<p>In a nutshel: when you compile multiple files, each file in the list is a compilation unit. For each compilation unit, there are multiple
files: the file itself and all included files from it. So, for each compilation unit, you should create a DICompileUnit and for all files in
that unit, you should create a DIFile attached to it.</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection">
<a name="type"><a href="http://llvm.org/doxygen/classllvm_1_1DIType.html">DIType</a></a>
</div>
<div class="doc_text">
<p>Types (variables, functions, etc) are represented in DIType variants, which is derived as follows: Base types (int, float, etc)
are represented as DIBasicType; pointers, consts, volatile, packed and typerefs as DIDerivedType; and all aggregate (arrays,
structs, classes) as well as functions as DICompositeType.</p>
<p><b>DIBasicType</b> can be constructed by using:</b>
<ol>
<li>The <b>context</b>, which is always the compilation unit, since base types are language specific.</li>
<li>The <b>name</b> of the type. In C/C++, these would be "int", "float", etc. depending on their types and sizes.</li>
<li>The <b>file</b> For base types, this is irrelevant.</li>
<li>The <b>line number</b> in that file, also irrelevant.</li>
<li>The <b>size</b> of the type, in bits.</li>
<li>The <b>alignment</b> of the type, in bits.</li>
<li>The <b>offset</b> of the type, in bits. For base types, this is generally zero.</li>
<li>The <b>flags</b> Platform specific. Put zero unless you know what you're doing.</li>
<li>The <b>Dwarf encoding</b> of the type. If it is boolean, float (single or double), signed (any size), etc.</li>
</ol>
<h5>Example:</h5>
<pre class="doc_code">// Ex. "int" on 32-bit architectures are generally:
DIBasicType int_type =
factory.CreateBasicType(
/* DIDescriptor */ context = cunit,
/* StringRef */ "int",
/* DIFile */ file, /* unsigned */ line,
/* unsigned */ size = 32, /* unsigned */ align = 32, /* unsigned */ offset = 0,
/* unsigned */ flags = 0,
/* unsigned */ DW_ATE_signed
);
</pre>
<p>Other, more complex types (arrays, structs, unions, classes, subroutines) can be created using the <code>DICompositeType</code>
using the right Dwarf TAG (DW_TAG_array_type, DW_TAG_structure_type, etc).</p>
<p>Types such as pointers, typerefs (const, volatile etc.), and struc/class members can be created using the <code>DIDerivedType</code>,
again with the appropriate TAGs (DW_TAG_pointer_type, DW_TAG_const_type, DW_TAG_member etc.)</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection">
<a name="global"><a href="http://llvm.org/doxygen/classllvm_1_1DIGlobalVariable.html">DIGlobalVariable</a></a>
</div>
<div class="doc_text">
<p>Each (source/header) file has its own global variables. Some variables are local to the unit (static), some are only declarations, etc. All
that information needs to encoded in the debug symbols.</p>
<p>Globals variables are represented by the DIGlobalVariable class:</p>
<ol>
<li>The <b>context</b> where it belongs. This could be the file itself, or some lexical block (ex. namespaces).</li>
<li>The <b>name</b> of the global variable as it appears in the source. Unnamed objects must have empty string.</li>
<li>The <b>display name</b> of the global variable. Use the same as in name.</li>
<li>The <b>linkage name</b> of the global variable. In C++ this is the mangled name. If no mangling is necessary, it's just the same as the name.</li>
<li>The <b>file</b> it belongs to. One of the created above, which could be a header file or the source file itself.</li>
<li>The <b>line</b> in the file above.</li>
<li>The <b>type</b> of the variable. See <a href="#type">DIType</a></li>
<li>If it's <b>local</b> to this compilation unit (static).</li>
<li>If it's <b>definition</b> as well as declaration.</li>
<li>The <b>global variable</b> it referst to. This is used to infer more information when lowering DWARF information.</li>
</ol>
<h5>Example:</h5>
<pre class="doc_code">// On source, this variable:
7: static int Foo = 42;
// Is created:
DIGlobalVariable gv =
factory.CreateGlobalVariable(
/* DIDescriptor */ context = cunit,
/* StringRef */ name = "Foo", display_name = "Foo", linkage_name = "Foo",
/* DIFile */ file, /* unsigned */ line = 7,
/* DIType */ type = int_type,
/* bool */ isLocal = true,
/* bool */ isDefinition = true,
/* GlobalVariable* */ global);
</pre>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection">
<a name="subprogram"><a href="http://llvm.org/doxygen/classllvm_1_1DISubprogram.html">DISubprogram</a></a>
</div>
<div class="doc_text">
<p>Subprograms are functions, procedures, member functions, etc. It's represented by DISubprogram, that needs the follownig information:</p>
<ol>
<li>The <b>context</b> where it belongs. This could be the file itself, or some lexical block (ex. namespaces).</li>
<li>The <b>name</b> of the function as it appears in the source.</li>
<li>The <b>display name</b> of the function. Same as above.</li>
<li>The <b>linkage name</b> of the function. In C++ this is the mangled name. Omit this field if no mangling is necessary.</li>
<li>The <b>file</b> it belongs to. One of the created above, which could be a header file or the source file itself.</li>
<li>The <b>line</b> in the file above.</li>
<li>The <b>return type</b> of the function. See <a href="#type">DIType</a></li>
<li>If it's <b>local</b> to this compilation unit (static).</li>
<li>If it's <b>definition</b> as well as declaration.</li>
<li>VK, VIndex, DIType t</li>
<li>If it was generated by the compiler, set <b>isArtificial</b> to true.</li>
<li>If it was optimized by the front-end, set <b>isOptimized</b> to true.</li>
<li>The <b>Function*</b> (Value*) it referst to. This is used to infer more information when lowering DWARF information. (ex. if it is variadic)</li>
</ol>
<h5>Example:</h5>
<pre class="doc_code">// In source, the function:
11: int my_function ( char* name, int size );
// Is created:
DISubprogram func =
factory.CreateSubprogram(
/* DIDescriptor */ context = cunit,
/* StringRef */ name = "my_function", name = "my_function", name = "my_function",
/* DIFile */ file, /* unsigned */ line = 11,
/* DIType */ type = int_type,
/* bool */ isLocal = false,
/* bool */ isDefinition = false,
VK, VIndex, t,
/* bool */ isArtificial = false,
/* bool */ isOptimized = false,
/* Function* */ function
);
</pre>
<p>This way, the function will appear on the list of global symbols, with proper return type and parameters, but
you also need to associate the parameters to local variables (See <a href="variable">DIVariable</a>) in order to see
their values while debugging.</p>
</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection">
<a name="variable"><a href="http://llvm.org/doxygen/classllvm_1_1DIVariable.html">DIVariable</a></a>
</div>
<div class="doc_text">
<p>.</p>
<p><b>DIVariable</b> can be constructed by using:</b>
<ol>
<li>The <b>Dwarf TAG</b> type, which defines the typeof variable (argument, auto, etc).</li>
<li>The <b>context</b>, which is always the compilation unit, since base types are language specific.</li>
<li>The <b>name</b> of the type. In C/C++, these would be "int", "float", etc. depending on their types and sizes.</li>
<li>The <b>file</b> For base types, this is irrelevant.</li>
<li>The <b>line number</b> in that file, also irrelevant.</li>
<li>The <b>type</b> of the variable. See <a href="#type">DIType</a></li>
<li>If it's to <b>preserve</b> the symbol.</li>
</ol>
<h5>Example:</h5>
<pre class="doc_code">// On source, this variable:
215: int Bar = 135;
// Is created:
DIVariable var =
factory.CreateVariable(
/* llvm_dwarf_constants */ dwarf::DW_TAG_auto_variable,
/* DIDescriptor */ context = lex_block,
/* StringRef */ "Bar",
/* DIFile */ file, /* unsigned */ line = 215,
/* DIType */ type = int_type,
/* bool */ preserve = true
);
</pre>
<p>Arguments are created the same way, but with <code>dwarf::DW_TAG_arg_variable</code>.</p>
<p>Once you got hold of the DIVariable and created the Value* for that variable, you have to declare the variable, and associate
the debug location to that declare, so the debugger can capture its value during run time. For that you have to get hold of the
variable itself (Value*), the debug information you just created (DIVariable) and the basic block you're going to declare the
variable. That will insert an <code>%llvm.dbg.declare({}*, metadata)</code> call in the block you're emitting code.</p>
<pre class="doc_code">// Declare
Instruction* instr =
factory.InsertDeclare(
/* Value* */ variable,
/* DIVariable */ debug_info,
/* BasicBlock* */ block);
</pre>
</div>
<!-- *********************************************************************** -->
<div class="doc_section"> <a name="accessing">Accessing Debug Information</a></div>
<!-- *********************************************************************** -->
<div class="doc_text">
TODO
</div>
<!-- *********************************************************************** -->
<div class="doc_section"> <a name="line_info">Debug Line Information</a></div>
<!-- *********************************************************************** -->
<div class="doc_text">
<p>With all program objects assigned to debug information, we can start thinking about line information. As you go along, stepping
the source, the debugger needs to know which instruction belongs to which source file/line and what was the context at that time,
to be able to print variables, etc.</p>
<p>To associate line information with IR instructions (including llvm.dgb.declare calls), you have to use the <code>DebugLoc</code>
class, as associate it to the instruction or the IRBuilder itself, which will then pass it to every instruction it creates.</p>
<p>It is important, then, that you keep the builder up-to-date regarding DebugLocation, so you can export the correct line information
and the debugger can step correctly through the user's code.</p>
<pre class="doc_code">// Creating a debug location:
DebugLoc dbg = DebugLoc::get(line, column, context);
// Associating with instructions
Instruction* instr = factory.InsertDeclare(...);
instr->setDebugLoc(dbg);
// Associating with the IRBuilder
IRBuilder<> irb(bb);
irb.SetCurrentDebugLocation(dbg);
// This will export the correct line information automatically
irb.CreateAloca(...);
</pre>
<p>Whatch out, DebugLoc's context is actually an MDNode*. Because DIDescriptor overloads the * operator, passing a DIDescriptor works
here, so don't be tempted to save the MDNode (to use it later?), as you'll lose the information of which type of debug information
it is.</p>
</div>
</div>
<!-- *********************************************************************** -->
<div class="doc_section"> <a name="path">Debug Info Path within LLVM</a></div>
<!-- *********************************************************************** -->
<div class="doc_text">
<p>The Debug Info has to be created by the front end (as it has the line information, as well as additional
type information that LLVM IR doesn't keep. This information, cerated with the help of the <a href="#helpers">DIFactory</a>,
has to be lowered by LLVM into ASM/object information.</p>
</div>
<!-- ======================================================================= -->
<div class="doc_subsection">
<a name="path-fe">Front-End: Clang/DebugInfo</a>
</div>
<div class="doc_text">
<p>A good example on how to tie up the debug information can be found at Clang's <code>tools/clang/lib/CodeGen/CGDebugInfo.h</code> and it's used by the
<code>tools/clang/lib/CodeGen/CodeGen*.cpp</code> files.</p>
<p>Also, the pass responsible for printing the debug information in the IR is <code>lib/Analysis/ModuleDebugInfoPrinter.cpp</code>.
</div>
<!-- ======================================================================= -->
<div class="doc_subsection">
<a name="path-be">Back-End: AsmPrinter</a>
</div>
<div class="doc_text">
<p>Once the debug information is encoded in metadata nodes (MDNodes and MDValues), LLVM lowers this information using
<code>lib/CodeGen/AsmPrinter/DwarfDebug.cpp</code>.</p>
</div>
<!-- *********************************************************************** -->
<div class="doc_section"> <a name="discussions">Further discussions</a></div>
<!-- *********************************************************************** -->
<div class="doc_text">
<h5>Polymorphism:</h5>
<p>There is a simple hierarchy in the classes, but mostly to aggregate functionality in the helper functions than to create a proper
polymorphism. In that sense, all DI* objects are passed by value (since they're normally the size of a pointer). That fact creates some
problems, as we'll see below.</p>
<p>To create DI* objects you should use the DIFactory (from <a href="http://llvm.org/doxygen/DebugInfo_8h_source.html">DebugInfo.h</a>).
You can create
the objects by directly calling their constructors, but that won't guarantee you get the parameters right. The DIFactory has some simple
methods for creating variables, functions, files, types and its variations that makes it much easier to deal with debug information.</p>
<p>Remember, the DI* classes are only wrappers to MDNodes, and MDNodes can easlily be passed around by themselves. It means that it could
happen that you pass a <i>DIType</i> MDNode and re-wrap it on a <i>DIFile</i>, which will then get the wrong information when using the
helper functions.</p>
<p>Although <i>DIDescripto</i> has an <code>isBasicType()</code> method, you're not obliged to use it when converting types, and you lose
that information anyway when you get the MDNode inside. And since DIDescriptors overload the <code>*</code> and <code>-></code> operators,
to use proper polymorphism, a major refactoring will be required.</p>
<h5>DIDescriptor as base type:</h5>
<p>It's here we see the first problem with not using polymorphism. DIDescriptor is used as a generic type for the context (which can be
files, functions, lexical blocks, variables) but you can't pass a DILexicalBlock in place of a DIDescriptor, unless you have an explicit
constructor (or some other copy mechanism) that allows it.</p>
<p>DIDescriptor also has <code>is'SubType'()</code> methods, to indicate their sub-types, so you can convert them back to their original type,
but for that, you need a block of <code>(if)/(else if)</code> statements, which is less than ideal. Still, the constructors are in place,
which allows us to do the basic debug information costruction in this text. We shall not need to cast the objects back to their original type.</p>
<p>As you can see, the context of a variable is the lexical block it belongs to, and its context is the subprogram, whose context is the file
(or another lexical block, if inside a namespace) and so on.</p>
<h5>Validation:</h5>
<p>It has been reported in the list that some debug data is generated incorrectly when using the DIFactory. Although it's impossible to
validate and fix all problems, a simple validation like the one done in IR could catch most of the simple errors that would, in teh back-end,
generate bad information.</p>
</div>
<!-- *********************************************************************** -->
<hr>
<address>
<a href="http://jigsaw.w3.org/css-validator/check/referer"><img src="http://jigsaw.w3.org/css-validator/images/vcss-blue" alt="Valid CSS"></a>
<a href="http://validator.w3.org/check/referer"><img src="http://www.w3.org/Icons/valid-html401-blue" alt="Valid HTML 4.01"></a>
<a href="mailto:renato.golin@arm.com">Renato Golin</a><br>
<a href="http://llvm.org">LLVM Compiler Infrastructure</a><br>
Last modified: Tue Aug 24 11:13:51 BST 2010
</address>
</body>
</html>