[cfe-dev] about AST of clang

[Manuel Klimek]

On Sat, Apr 21, 2012 at 2:55 AM, Ronan Keryell
<Ronan.Keryell at wild-systems.com> wrote:
>> On Fri, Apr 20, 2012 at 9:57 PM, Douglas Gregor <dgregor at apple.com> wrote:
>
>    >> The Clang AST is not designed to be edited. It can be done, but
>    >> ensuring that the resulting AST is well-formed is a non-trivial
>    >> task.
>
>>>>>> On Sat, 21 Apr 2012 00:33:55 +0200, Manuel Klimek <klimek at google.com> said:
>
>    Manuel> And to propose an alternative solution (which might of
>    Manuel> course not apply to the situation at hand): we've had great
>    Manuel> success with identifying nodes in the AST that we want to
>    Manuel> change, mapping it to source code, doing text
>    Manuel> transformations on the source code, and then compiling the
>    Manuel> resulting source code back into an AST / an executable.
>
> Because of a lack of resources, we did this more than 15 years ago in
> another compiler we work on and it is still there. Now I really
> understand in the real life this concept:
> http://en.wikipedia.org/wiki/Technical_debt
> ;-)

Well, I disagree that it's a poor architecture, which would not make
it technical debt ;) Instead, I suggest it's a hard problem and it
will take a decent amount of maintenance effort no matter what you do.

> So, yes it works on some simple cases... But it is a nightmare to have
> to apply patches on a kludge when you have to improve the parser or
> change the internal representation only a little bit or to deal with
> more general cases.

As I said before, I might not understand what you're trying to do.
That said, I think that changing the code is a superset of changing
the AST, so I don't understand why it's harder to do certain things
that way. It of course requires a very precise mapping of AST nodes to
source code, which clang luckily enough has. It also requires that you
still want to do C++ (if you don't, that would be a case where what I
say clarly does not apply).

As for changes of internal representation getting in your way - how's
that better when you directly work on the AST - on the contrary, I
expect subtle changes to the invariants of the AST to make it much
harder to still produce correct ASTs by shoving around AST nodes,
instead of making textual changes.

And third, if you ever want the changes to go back to the programmer
in code form, you suddenly need to care about formatting etc, and
minimally disruptive changes to the text.

> That is why we are thinking to add support to Clang to have a
> cleaner way to have source-to-source transformations that preserve
> comments and so on.

I don't understand yet why you think the direct tree transformation is
per se cleaner.

Cheers,
/Manuel