[cfe-dev] About AST rewriting / manipulation

[Simone Pellegrini]

Yes, the Rewriter does the job!
But the main problem with the Rewriter is that any change I apply to the 
syntax tree (let's say... insert text or node replacement) are not kept 
in the original syntax tree.

To me the Rewriter seems like a buffer that keeps track of the changes I 
made and when I say to print it back it merges the syntax tree with the 
changes I have provided. Is that correct? So, I will explain my 
problem... I have a source file and I want to analyze it and then apply 
(if it's worth doing it) a list of transformations considering a 
particular order! So... I want to be able to apply several 
transformations on the same AST. Using the rewriter it seems that I can 
apply one transformation... then produce a source file... parse it 
again... analyze and apply another transformation... and so on!
And that's it's real inefficient! :(

I guess I could mix the usage of the TokenRewriter and the prettyPrint() 
method to rewrite portions of the AST I have changed... or have a look 
at the Rewriter implementation because as now it's not clear to me how 
it works! :)

regards, S. Pellegrini

Douglas Gregor wrote:
> On Feb 2, 2009, at 8:40 AM, Simone Pellegrini wrote:
>> Thanks for the response,
>> I've chosen the simpler way, I just create clang nodes from scratch 
>> and replace / insert these nodes with the old one in the syntax 
>> tree!  At the end it's not so complex as I thought... but now I have 
>> another problem! I want to write back the modified syntax tree but no 
>> matter which kind of changes I made to the syntax tree... when I use 
>> the TokenRewriter:
>>
>> const LangOptions &LangOpts = Ctx.getLangOptions();
>> TokenRewriter Rewriter(Ctx.getSourceManager().getMainFileID(), 
>> Ctx.getSourceManager(), LangOpts);
>>
>> // Print out the output.
>> for (TokenRewriter::token_iterator I = Rewriter.token_begin(), E = 
>> Rewriter.token_end(); I != E; ++I)
>>       out << pp.getSpelling(*I);
>>
>> I still get the original code!
>
> As I understand it, the TokenRewriter is only meant to be used to 
> insert tokens into the token stream as it's being rewritten. It's not 
> designed for AST-level rewrites.
>
>> How can I rewrite a syntax tree to Source file? Is there something in 
>> the API that I am missing? Should I use the Rewriter? If yes, why the 
>> rewriter doesn't provide an InsertStmt method? With the current API I 
>> can add text... and replace statements... what about inserting 
>> statements? and... remove statements?
>
> I believe the Rewriter can do what you want, but it will take some 
> effort. The rewriter allows you to insert/remove/replace text in the 
> source file with any other text, using SourceLocations to specify 
> where in the source file the modifications will be made. That's one 
> approach to AST-based rewriting, since one can turn a 
> statement/expression/declaration into a string and then insert it at 
> the appropriate place in the source.
>
> Clang has some functionality for pretty-printing statements and 
> expressions as source code, but we haven't put a lot of effort into 
> making sure that the result is well-formed C++. Depending on how 
> complicated your statements and expressions are, this might take no 
> work or it might take a lot of work.
>
>     - Doug