[cfe-dev] Dumping AST information to other formats

[Aaron Ballman via cfe-commits]

On Mon, Nov 26, 2018 at 2:27 PM Keane, Erich via cfe-commits
<cfe-commits at lists.llvm.org> wrote:
>
> I'd be tentative to commit to any stability guarantees, particularly as the AST tends to change reasonably often.

Understood. This presupposes that there are stability guarantees to be
made surrounding the information provided by the AST, but I really
think those guarantees exist. We add new things to the AST, but we
rarely remove entire AST nodes or refactor AST nodes such that
critical information is no longer exposed. Most often, we reformulate
how information is exposed in the AST, modify enumerations, change
underlying data types, etc, but that should (ideally) not impact the
output of this feature.

>  That said, I can see the value of this.
>
> Additionally, it would be preferential I suspect if we could make the standard ast-dump just another (albeit default) "format" in whatever plugin architecture you plan.  I presume that would result in a lesser maintenance burden (again, depending on the plugin architecture).

The syntax I was proposing is: -ast-dump, -ast-dump=format where
format can either be "default" or "json" (but maybe someday we decide
we need "xml" or "s-expr", etc). I wasn't envisioning a plugin for
this; I was proposing to modify the existing -ast-dump option to
optionally support specifying the format, then threading that format
information through to the action that dumps the AST out to a file in
order to pick the proper AST dumping interface.

~Aaron

>
> -----Original Message-----
> From: cfe-dev [mailto:cfe-dev-bounces at lists.llvm.org] On Behalf Of Aaron Ballman via cfe-dev
> Sent: Monday, November 26, 2018 11:20 AM
> To: cfe-dev <cfe-dev at lists.llvm.org>; cfe-commits <cfe-commits at lists.llvm.org>
> Cc: Eric Schulte <eschulte at grammatech.com>
> Subject: [cfe-dev] Dumping AST information to other formats
>
> Clang currently supports various -cc1 options that allow displaying AST information (-ast-dump, -ast-print, -ast-list, etc), but these options are not convenient to consume by third-party tools. GrammaTech has ongoing research efforts where we would like to output some information from the AST to a more open machine-consumable format (such as JSON or s-expressions). We propose adding an optional output format to the -ast-dump command allowing the user to select from either the default or JSON formats. If the output format is not explicitly specified, it will continue to default to the same textual representation it uses today. e.g., clang -cc1 -ast-dump=json foo.c.
> This feature is intended to output a safe subset of AST information that is considered crucial rather than an implementation detail (like the name of a NamedDecl object and the SourceRange for the name), so the output is expected to be mostly stable between releases.
>
> Once upon a time, there was -ast-print-xml. This -cc1 option was dropped because it was frequently out of sync with the AST data. It is right to ask: why would JSON, etc be any different? This is still an open question, but a goal of this implementation will be to ensure it's easier to maintain as the AST evolves. However, this feature is intended to output a safe subset of AST information, so I don't think this feature will require any more burden to support than -ast-dump already requires (which is extremely limited). If AST information is found to be missing from the output, it seems reasonable to have a discussion as to whether it is stable information or an implementation detail, so missing information is to be expected rather than concerned by. That said, GrammaTech is able to commit to maintaining this code for at least the next 1-2 years and possibly beyond as it useful functionality for our research efforts.
>
> I wanted to see if there were concerns or implementation ideas the community wanted to share before beginning the implementation phase of this feature.
>
> ~Aaron
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits