[cfe-dev] clang AST printer

[Alexander Kornienko]

+dgregor

On Mon, Sep 24, 2012 at 5:02 AM, Philip Craig <philipjcraig at gmail.com>wrote:

> On Sat, Sep 22, 2012 at 7:00 AM, Philip Craig <philipjcraig at gmail.com>
> wrote:
> > On Fri, Sep 21, 2012 at 11:21 PM, Alexander Kornienko <alexfh at google.com>
> wrote:
> >>
> >> On Fri, Sep 21, 2012 at 2:40 PM, Manuel Klimek <klimek at google.com>
> wrote:
> >>>
> >>> On Fri, Sep 21, 2012 at 12:32 PM, Philip Craig <philipjcraig at gmail.com
> >
> >>> wrote:
> >>> > On Fri, Sep 21, 2012 at 7:49 PM, Manuel Klimek <klimek at google.com>
> >>> > wrote:
> >>> >> Hi Philip,
> >>> >>
> >>> >> we had a discussion around the strategy for handling ast dumping
> >>> >> fairly recently. I think the common agreement in the end was that
> >>> >> instead of having an extra tool, we want to make clang's -ast-dump
> >>> >> awesome.
> >>> >
> >>> > Was this discussion on the mailing list? Any chance you could point
> me
> >>> > to it?
> >>>
> >>>
> >>>
> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20120716/060831.html
> >
> > Thanks.
> >
> >>> >> I don't know what your ultimate goal is here, but from my
> >>> >> side any work that goes towards making clang -ast-dump better would
> be
> >>> >> highly appreciated :)
> >>> >
> >>> > My motivation was having some way of getting the structure of the AST
> >>> > for a given piece of source code, so that it is easier to correctly
> >>> > call the AST matchers to match that code. The issue with clang
> >>> > -ast-dump is that it pretty prints decls and types. So would the goal
> >>> > here be to add more command line options to clang to control how
> >>> > -ast-dump prints the AST?
> >>>
> >>> No, I think the goal is to dump decls and types in a sensible way :)
> >>
> >>
> >> More specifically, I think we agreed on changing -ast-dump from
> >> pretty-printing declarations to outputting them in the same LISP-style
> >> format, which is used for statements. I was going to implement that, but
> >> never had time to do this. If you're going to continue work on your
> utility,
> >> it would be much more valuable, if you instead improved clang's current
> >> -ast-dump option. In that case you consider to go this way, here's
> where to
> >> start:
> >>
> >> clang/lib/Frontend/ASTConsumers.cpp:120: ASTConsumer
> >> *clang::CreateASTDumper(StringRef FilterString)
> >>
> >> is a common entry point, used by "clang -cc1 -ast-dump" and by
> "clang-check
> >> -ast-dump". An additional benefit from this would be that after
> "-ast-dump"
> >> starts outputting the AST for declarations in a structured form,
> >> "-ast-dump-xml" will become useless and can be removed.
> >
> > I'll have a look into that and get an idea of what's involved.
>
> I'm going to try working on this. Here's my thoughts on how to structure
> this.
>
> Currently, it looks like this (I'm ignoring types for now):
>
> CreateASTDumper calls either Decl::print() or Decl::dump().
>
> Both Decl::print() and Decl::dump() use class DeclPrinter. The only
> difference between these uses is that Decl::dump() passes a
> PrintingPolicy that has DumpSourceManager set.
>
> When DeclPrinter encounters a statement, it calls Stmt::printPretty().
> If DumpSourceManger is set, then Stmt::printPretty() calls
> Stmt::dump(), otherwise it uses class StmtPrinter.
>
> Stmt::dump() uses class StmtDumper.
>
>
> My proposed changes are:
>
> I will change Stmt::printPretty() to never call Stmt::dump(). This
> also removes the need for the DumpSourceManager field in
> PrintingPolicy.
>
> I will create a new DeclDumper class, which is called by Decl::dump(),
> and which calls Stmt::dump() when it encounters a statement.
>
> There will be some duplication of functionality and state between
> StmtDumper and DeclDumper. So I will extract parts of StmtDumper out
> into an ASTDumper class. This will result in a design that is very
> similar to the ASTWriter for serialization.
>

As for me, this looks fine, but I'm relatively new to Clang, so I can
easily miss something. It would be great, if Doug looked at this.


> My first patch will be to create the ASTDumper class. Later patches
> will create DeclDumper. For the creation of the DeclDumper class,
> should I do this as one big patch, or should I try to find a way to
> break it into smaller patches?
>

As long as you can split this into self-contained patches, which have value
on their own, please do. But if your changes will only have value together,
I guess they are better submitted as one patch. Doug, am I right here?

Anyway, you can post your patches as you go, so you can get feedback
earlier. I would also strongly suggest using our code review tool to submit
patches: http://llvm-reviews.chandlerc.com/

>>>
> >>> > Are there any other areas where you think -ast-dump needs to be
> better?
> >>>
> >>> As you said, decls and types need to be structured nicely. Also, a
> >>> while ago Richard proposed a patch to add coloring, no idea how far he
> >>> got (cc'ing him)
> >>>
> >>> Cheers,
> >>> /Manuel
> >>>
> >>> >
> >>> >>
> >>> >> Cheers,
> >>> >> /Manuel
> >>> >>
> >>> >> On Fri, Sep 21, 2012 at 1:17 AM, Philip Craig <
> philipjcraig at gmail.com>
> >>> >> wrote:
> >>> >>> Hi,
> >>> >>>
> >>> >>> I've been writing a tool to print a clang AST. You can find it at
> >>> >>> https://github.com/philipc/clang-ast. I've been mostly writing
> this as
> >>> >>> a learning exercise, but I would like to see if there is any wider
> >>> >>> interest in the tool.
> >>> >>>
> >>> >>> I'm new to clang, and interested in working on tooling. For this
> task,
> >>> >>> it is important to have an understanding of the AST. I've found the
> >>> >>> documentation at http://clang.llvm.org/docs/InternalsManual.htmland
> >>> >>> http://clang.llvm.org/docs/IntroductionToTheClangAST.html (is
> there
> >>> >>> any more?), but I think it would be helpful to be able to print the
> >>> >>> AST for source code to see how it is used in practice. I've found
> two
> >>> >>> ways to do this currently, which aren't quite what I wanted:
> >>> >>>
> >>> >>> clang --ast-dump
> >>> >>> - pretty prints some parts, has too much internal info, more suited
> >>> >>> for debugging use by clang developers
> >>> >>>
> >>> >>> clang --ast-dump-xml
> >>> >>> - incomplete and XML is too verbose for this purpose
> >>> >>>
> >>> >>> Since RecursiveASTVisitor seems to be the API that will be used by
> >>> >>> tool developers, I've been using RAV to print the AST. I've found a
> >>> >>> few minor bugs in RAV as a result of this, which I'm in the
> process of
> >>> >>> submitting.
> >>> >>>
> >>> >>> One limitation of RAV is that the Visit methods aren't given
> >>> >>> information about their relationship with their parent, so the tool
> >>> >>> can only list all the children of a node, without distinguishing
> >>> >>> between the LHS and RHS of an operator, for example. Is there any
> >>> >>> desire to extend RAV to be able to do this?
> >>> >>>
> >>> >>> I've been writing small code snippets to test printing the various
> >>> >>> parts of the AST. If you want to see examples of the output of the
> >>> >>> tool, this is included in the test cases. As a quick example, "void
> >>> >>> foo() {}" is printed as:
> >>> >>>
> >>> >>>   FunctionDecl
> >>> >>>     DeclarationName foo
> >>> >>>     FunctionNoProtoType
> >>> >>>       BuiltinType void
> >>> >>>     CompoundStmt
> >>> >>>
> >>> >>> Finally, I've been trying to give a textual description of the AST
> >>> >>> grammar in
> https://github.com/philipc/clang-ast/blob/master/ast.txt.
> >>> >>> This is an attempt to give something similar to
> >>> >>> http://docs.python.org/py3k/library/ast.html#abstract-grammar.
> I'm not
> >>> >>> sure if it is turning out to be that useful though.
> >>> >>>
> >>> >>> Any comments welcome!
> >>> >>>
> >>> >>> Thanks,
> >>> >>> Philip
>


-- 
Regards,
Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20120924/8ffcf1d3/attachment.html>