[cfe-dev] Future of AST transformation

[Richard Smith]

On Thu, Jul 19, 2012 at 1:36 AM, Ronan Keryell <
Ronan.Keryell at wild-systems.com> wrote:

> >>>>> On Wed, 18 Jul 2012 15:22:46 -0700, Richard Smith <
> richard at metafoo.co.uk> said:
>
>     Richard> Since you said you're performing source to source
>     Richard> transformations, AST transformation is unlikely to be a
>     Richard> good path to follow. A source to source transformation tool
>     Richard> should usually make very targeted modifications to the
>     Richard> code, and it's not really practical to deduce what those
>     Richard> changes should have been if all you have are ASTs from
>     Richard> before and after (think about preserving whitespace,
>     Richard> comments, macros, templates, ...).
>
> It depends on what you want to do with source-to-source transformation.
>
> If for example I use source-to-source transformations to generate from a
> sequential C/C++ program a parallel program with multiple processes
> communicating with MPI, with OpenMP #pragma on each SMP node with some
> calls to SIMD intrinsics and some parts in CUDA or OpenCL to address
> heterogeneous computing, a simple textual transformation is just NOT a
> good path to follow.
>
> Another typical use case is high-level hardware synthesis from C/C++.
>

Right, if you're generating input to another tool, emitting output from a
parsed AST is completely reasonable. My message was mostly concerning tools
whose output is supposed to be maintained by a human; apologies for not
making that clear.


>     Richard> The usual approach for clang-based source-to-source
>     Richard> transformation tools is to use the AST to determine what
>     Richard> changes should be made, then produce a list of
>     Richard> modifications to be made to the original source file. See
>     Richard> clang::Rewriter, clang::tooling::Replacement, and
>     Richard> clang::tooling::RefactoringTool for some components which
>     Richard> make it easier to build such tools.
>
> Yes, but some people are interested by a less "usual approach". :-)
> For serious things, AST transformations are the only way to go.
>
> Keeping track of comments in the AST is a first step, even if
> source-to-source transformations are intractable in the general case
> because you need to keep the semantics of... comments. I do not even
> talk about macros...
>
> Just imagine how to translate
> /* ... */
> /* ... */ a/*  */
>     +=
>   /* ... */b  /* ... */
>  ; //...
>
> to
> a = a + b;
>
> Where do you want to keep/duplicate the comments ? You have to
> "understand" them to resynthesize their new layout and content. :-(
>
> Even with simpler cases it is already a nightmare...
>

Exactly; this is one reason why AST to AST transformations for such cases
are problematic. If your output is a list of source transformations,
though, it's easy to produce

/* ... */
/* ... */ a/*  */
    = a +
  /* ... */b  /* ... */
 ; //...

or

/* ... */
/* ... */ a/*  */
    =
  /* ... */a + b  /* ... */
 ; //...

as you prefer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20120719/150dd68e/attachment.html>