[lldb-dev] ASTImporter usage

[Sean Callanan]

Henry,

I'd be happy to give you a quick overview.

–

In order to optimize compilation time and memory usage, Clang keeps Types, Decls, and Stmts in ASTContexts.  ASTContexts overload the placement new operator with a custom bump allocator that allows these objects to be associated tightly with them, and allow a single free to destroy everything associated with that ASTContext.

As a side effect of this structure, ASTContexts are insert-only.  This means that they will grow in size over time, and if you ever want to have "temporary" Types/Decls, they will need to live in their own ASTContext and deleted as a unit.

The Clang parser assumes that all declarations and types it deals with are in a single ASTContext.

LLDB loads debug information from many files and wants to be able to unload it as well.  That means that we create an independent AST context for each symbol file, and create/delete them at will.  This is great for memory usage, and our own type handling functions all know how to deal with types from different AST contexts.

We embed Clang into LLDB to do our expression parsing, though, and when an expression uses a Decl or a Type that lives in a symbol file, that Decl or Type needs to be moved to the ASTContext that Clang is using for parsing.  This is a "deep copy" – everything that the Type or Decl refers to needs to be potentially moved into the parser's ASTContext.

That's where the ASTImporter comes in.  The ASTImporter knows how to "deep copy" entities between ASTContexts.  In addition, it has a "minimal" mode where it allows us to provide particular chunks of the abstract syntax graph on demand rather than deep copying everything.  LLDB relies heavily on this "minimal" mode and when it doesn't work the Clang parser can crash or produce bizarre parse errors.

LLDB actually has several types of ASTContexts:

(1)	the ASTContexts for entities in symbol files, one per symbol file;
(2)	the ASTContexts to parse an expression, one per expression; and
(3)	the scratch ASTContexts, which contains types and Decls that were created by the user (for example, by declaring a type in an expression) but need to persist beyond the lifetime of the expression (for example, if the user created an expression result variable of that type).

LLDB's use of AST importers is managed by the ClangASTImporter class.  It maintains origin information for all Decls in ASTContexts it manages, and dynamically allocates Minions to handle copying between specific ASTContexts.  Inside the ClangASTImporter class is a custom subclass of ASTImporter called Minion.  ClangASTImporter uses Import(Decl *) and Import(QualType) directly on its Minions, which call through to much of the rest of the ASTImporter interface.  Each minion also overloads Imported(), which is a callback that indicates what portions of the AST have been copied.  When they receive this callback, they update the ClangASTImporter keep the origin information up to date.

–

Sean
 
On Aug 16, 2012, at 10:15 AM, habbey at cs.wisc.edu wrote:

> Hello,
> 
> I'm a graduate student at the University of Wisconsin Madison and am
> beginning work on a static analyzer using clang.  I've been looking at the
> ASTImporter class of clang and noticed that it doesn't seem to be complete
> and isn't used directly by any part of clang, but that LLDB does use it. 
> I was wondering if anyone could help me understand what parts of the
> ASTImporter class LLDB uses and for what overall purpose.  If anyone has
> even a little bit of insight on this topic it'd be much appreciated.
> 
> Thank you
> 
> --henry abbey
> 
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev