<div dir="ltr"><div class="gmail_default" style="font-family:verdana,sans-serif">Thanks - Now I have little bit understanding of how the lldb parser works. I like the way it explained with the simple example(a+2).</div><div class="gmail_default" style="font-family:verdana,sans-serif">
<br></div><div class="gmail_default" style="font-family:verdana,sans-serif">It would be great if it can be extended and made available as a doc in <a href="http://llvm.org/docs/" style="font-family:arial">http://llvm.org/docs/</a> (or in similar place).</div>
<div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Regards</div><div class="gmail_default" style="font-family:verdana,sans-serif">
Samuel</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Aug 16, 2013 at 6:12 PM, Sean Callanan <span dir="ltr"><<a href="mailto:scallanan@apple.com" target="_blank">scallanan@apple.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">This is the outline of a brief presentation I gave on the LLDB expression parser.<div>
I’ve included some “thorny issues;” if we resolve these, the expression parser will get a lot better.</div><div>Please let me know if you have any questions.</div><div><br></div><div><div><b>Class layout</b></div><div><span style="white-space:pre-wrap"> </span>The master - ClangExpressionParser manages Clang and LLVM to compile a single expression</div>
<div><span style="white-space:pre-wrap"> </span>Its minions:</div><div><span style="white-space:pre-wrap"> </span>ClangExpression - a unit of parseable code</div><div><span style="white-space:pre-wrap"> </span>ClangUserExpression - specialized for the case where we’re using the “expr” command</div>
<div><span style="white-space:pre-wrap"> </span>ExpressionSourceCode - handles wrapping</div><div><span style="white-space:pre-wrap"> </span>ClangASTSource - resolves external variables</div><div><span style="white-space:pre-wrap"> </span>ClangExpressionDeclMap - specialized for the current frame (if stopped at a particular location in the program being debugged)</div>
<div><span style="white-space:pre-wrap"> </span>IRForTarget - rewrites IR</div><div><span style="white-space:pre-wrap"> </span>ASTResultSynthesizer - makes the result</div><div><span style="white-space:pre-wrap"> </span>IRMemoryMap - manages memory that may or may be in the program being debugged, or may be simulated by LLDB</div>
<div><span style="white-space:pre-wrap"> </span>IRExecutionUnit - specialized to be able to interact with the JIT</div></div><div><br></div><div><b>Basic Expression Flow</b></div><div><span style="white-space:pre-wrap"> </span>User enters the expression: (lldb) expr a + 2</div>
<div><span style="white-space:pre-wrap"> </span>We wrap the expression: void expr(arg *) { a + 2; }</div><div><br></div><div><span style="white-space:pre-wrap"> </span>We wrap differently based on expression context.</div>
<div><span style="white-space:pre-wrap"> </span>If stopped in a C++ instance method, we wrap as $__lldb_class::$__lldb_expr(void *)</div><div><span style="white-space:pre-wrap"> </span>If stopped in an Objective-C instance method, we wrap as an Objective-C category</div>
<div><span style="white-space:pre-wrap"> </span>If stopped in regular C code, we wrap as $__lldb_expr(void*)</div><div><span style="white-space:pre-wrap"> </span>But we always parse in Objective-C++ mode.</div><div><br>
</div><div><span style="white-space:pre-wrap"> </span>Typical wrapped expression:</div><div><span style="white-space:pre-wrap"> </span>#define … // custom definitions provided by LLDB or the user</div><div><span style="white-space:pre-wrap"> </span>void</div>
<div><span style="white-space:pre-wrap"> </span>$__lldb_class::$__lldb_expr // __lldb_class resolves to the type of *this in the current frame</div><div><span style="white-space:pre-wrap"> </span>(void *$__lldb_arg)</div>
<div><span style="white-space:pre-wrap"> </span>{</div><div><span style="white-space:pre-wrap"> </span>// expression text goes here</div><div><span style="white-space:pre-wrap"> </span>}</div><div><br></div><div><span style="white-space:pre-wrap"> </span>We resolve externals: “a” => int &a;</div>
<div><br></div><div><span style="white-space:pre-wrap"> </span>This happens via a question-and-answer process with the Clang compiler through the clang::ExternalASTSource interface</div><div><span style="white-space:pre-wrap"> </span>FindExternalVisibleDeclsByName searches for “globals” (globals from the perspective of the expression; these may be locals in the current stack frame)</div>
<div><span style="white-space:pre-wrap"> </span>FindExternalLexicalDecls searches a single struct for all entities of a particular type</div><div><span style="white-space:pre-wrap"> </span>CompleteType ensures that a single struct has all of its contents</div>
<div><span style="white-space:pre-wrap"> </span>(These are useful because we lazily complete structs, providing a forward declaration first and only filling it in when needed)<span style="white-space:pre-wrap"> </span></div>
<div><br></div><div><span style="white-space:pre-wrap"> </span>clang::ASTImporter is responsible for transferring Decls from one ASTContext (e.g., the ASTContext for a DWARF file) to another (e.g., the AST context for an expression)</div>
<div><span style="white-space:pre-wrap"> </span>Our ClangASTImporter manages many of these (“Minions"), because there are many separate DWARF files containing debug information.</div><div><span style="white-space:pre-wrap"> </span>We need to be able to remember where things came from.</div>
<div><br></div><div><span style="white-space:pre-wrap"> </span>We add the result: static int ret = a + 2;</div><div><br></div><div><span style="white-space:pre-wrap"> </span>This happens at the Clang AST level</div><div>
<span style="white-space:pre-wrap"> </span>We handle Lvalues and Rvalues differently.</div><div><span style="white-space:pre-wrap"> </span>For Lvalues, we store a pointer to them: T *$__result_ptr = …</div><div><span style="white-space:pre-wrap"> </span>For Rvalues, we store the value itself: static T $__result = … // static ensures the expression doesn’t try to use a register or something silly like that</div>
<div><span style="white-space:pre-wrap"> </span>We also store persistent types at this stage, e.g. struct $my_foo { int a; int b; }</div><div><br></div><div><span style="white-space:pre-wrap"> </span>We rewrite the IR: *(arg+0) = *(arg+8)+2</div>
<div><br></div><div><span style="white-space:pre-wrap"> </span>The IR as emitted by Clang’s CodeGen expects all external variables to be in symbols</div><div><span style="white-space:pre-wrap"> </span>This is inconvenient if they are e.g. in registers, since you can’t link against a register</div>
<div><span style="white-space:pre-wrap"> </span>This is also inconvenient for expression re-use, for example as a breakpoint condition… we’d have to re-link each time</div><div><span style="white-space:pre-wrap"> </span>Our solution is to indirect variables through a struct passed into the expression (void *$__lldb_arg)</div>
<div><br></div><div><span style="white-space:pre-wrap"> </span>Materializer’s job is to put all variables that aren’t referred to by symbols into this struct</div><div><span style="white-space:pre-wrap"> </span>It will create temporary storage as necessary (e.g., to hold a variable value that was in a register)</div>
<div><span style="white-space:pre-wrap"> </span>After the expression runs, a Dematerializer takes down all temporary storage, and ensures that variables are updated to reflect the expression’s side effects</div><div><br>
</div><div><span style="white-space:pre-wrap"> </span>The IRForTarget class does various cleanup to help RTDyldMemoryManager (ideally much of this shouldn’t be necessary)</div><div><span style="white-space:pre-wrap"> </span>It resolves all external symbols to avoid forcing RTDyldMemoryManager to resolve symbols</div>
<div><span style="white-space:pre-wrap"> </span>It creates a string and float literal pool so RTDyldMemoryManager doesn’t have to relocate the constant pool</div><div><span style="white-space:pre-wrap"> </span>It strips off nasty Objective-C metadata so RTDyldMemoryManager doesn’t have to look at it</div>
<div><br></div><div><span style="white-space:pre-wrap"> </span>We interpret or execute the result: (int)$0 = 6</div><div><br></div><div><span style="white-space:pre-wrap"> </span>IRExecutionUnit contains a module and the (real or simulated) memory it uses</div>
<div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>IRInterpreter can interpret a module without ever running the underlying process</div><div><span style="white-space:pre-wrap"> </span>It emulates IR instructions one by one</div>
<div><span style="white-space:pre-wrap"> </span>It uses lldb_private::Scalar to hold intermediate values, which is kinda limiting (no vectors, no FP math)</div><div><span style="white-space:pre-wrap"> </span>IRExecutionUnit simulates memory allocation etc. so we can do a lot of pointer magic</div>
<div><br></div><div><span style="white-space:pre-wrap"> </span>If the IRInterpreter can’t run, the MCJIT produces machine code and LLDB runs it</div><div><span style="white-space:pre-wrap"> </span>IRExecutionUnit vends a custom JITMemoryManager implementation</div>
<div><span style="white-space:pre-wrap"> </span>It remembers memory allocations and where functions were placed</div><div><span style="white-space:pre-wrap"> </span>After JIT, all sections are placed into the target and we report their new locations with mapSectionAddress</div>
<div><br></div><div><b>Selected Thorny Issues (concentrating on JIT-related issues)</b></div><div><span style="white-space:pre-wrap"> </span>Make the MCJIT more robust so we can rely on it more</div><div><span style="white-space:pre-wrap"> </span>Support all Mach-O and ELF relocation types</div>
<div><span style="white-space:pre-wrap"> </span>Don’t assume resolved symbols are in the current process</div><div><span style="white-space:pre-wrap"> </span>Don’t assume addresses fit into void*s</div><div><span style="white-space:pre-wrap"> </span>Make the IRInterpreter support all data types and instructions</div>
<div><span style="white-space:pre-wrap"> </span>Completely replace the LLVM interpreter!</div><span class="HOEnZb"><font color="#888888"><div><br></div><div><div>
<div>Sean</div>
</div>
<br></div></font></span></div><br>_______________________________________________<br>
lldb-dev mailing list<br>
<a href="mailto:lldb-dev@cs.uiuc.edu">lldb-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev</a><br>
<br></blockquote></div><br></div>