Ping?<br><br><div class="gmail_quote">On Thu, Jun 2, 2011 at 10:32 AM, Manuel Klimek <span dir="ltr"><<a href="mailto:klimek@google.com">klimek@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div>+cfe-dev, -cfe-commits</div><div><br></div><div>As requested by Chris I have reverted the Tooling stuff, and am bumping this thread on cfe-dev.</div><div><br></div><div>Please find the original mail proposing the patch sent to cfe-commits on May 23rd inlined below.</div>
<div><br></div><div>References to the original threads:</div><div>Proposal: <a href="http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20110523/042163.html" target="_blank">http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20110523/042163.html</a></div>
<div>Commit: <a href="http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20110530/042375.html" target="_blank">http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20110530/042375.html</a></div><div><br></div>
<div>Patches produced by the example tool sent out (has not got feedback yet):</div>
<div>Clang: <a href="http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20110523/042212.html" target="_blank">http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20110523/042212.html</a></div><div>LLVM: <a href="http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20110523/121387.html" target="_blank">http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20110523/121387.html</a></div>
<div>
<br></div><div>More context on why this is useful: when writing code transformation tools to do large scale changes on libraries with clang, we noticed that a lot of the code we wrote was "unrolled" matching of tree structures on the AST, which required a lot of dynamic casting and deeply nested control flow structures.</div>
<div>The library we implemented based on those findings allows a very declarative way of expressing interesting parts of the AST that you want to work on, with a callback mechanism to have an imperative part that does the actual transformation. The way this library uses templates to build up a reverse hierarchy of the clang AST allows us to expand the library of matchers without needing to manually copy the structure of the AST while still providing full compile time type checking.</div>
<div>We have used this library for many internal transformations (one more broadly applicable of them being included as an example tool), and Nico may be able to explain more how he's using the infrastructure for Chromium.</div>
<div><br></div><div>The future work (as outlined to some extend in the original email) is to build up code transformation tools (including the standard refactoring tools we know from other languages) and libraries to make it easy for developers to contribute both generally useful tools and make it easy to write one-off tools for cleanup transformations on larger code bases.</div>
<div><br></div><div>I would love this to be part of clang mainline, as the expertise and feedback of the community has been very helpful and valuable.</div><div><div></div><div class="h5"><div><br></div><div class="gmail_quote">
---------- Forwarded message ----------<br>
From: <b class="gmail_sendername">Manuel Klimek</b> <span dir="ltr"><<a href="mailto:klimek@google.com" target="_blank">klimek@google.com</a>></span><br>Date: Mon, May 23, 2011 at 3:03 PM<br>Subject: Patch: AST Matcher Framwork and an example tool<br>
To: <a href="mailto:cfe-commits@cs.uiuc.edu" target="_blank">cfe-commits@cs.uiuc.edu</a><br>Cc: Nico Weber <<a href="mailto:thakis@chromium.org" target="_blank">thakis@chromium.org</a>>, Marcin Kowalczyk <<a href="mailto:qrczak@google.com" target="_blank">qrczak@google.com</a>>, James Dennett <<a href="mailto:jdennett@google.com" target="_blank">jdennett@google.com</a>><br>
<br><br>This patch implements an AST matching framework that allows to write tools that match on the C++ ASTs. The main interface is in ASTMatchers.h, an example implementation of a tool that removes redundant .c_str() calls is in the example RemoveCStrCalls.cpp (patch to llvm/clang produced by running this tool will be sent out shortly in an extra email).<br>
<br>Currently we have an in-language DSL that allows to write expressions such as (taken from the .c_str() example):<br><span style="font-family:Arial, Helvetica, sans-serif;font-size:13px"><table cellpadding="0" cellspacing="0" style="padding-top:5px;padding-right:5px;padding-bottom:5px;padding-left:5px">
<tbody><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">ConstructorCall(
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
HasDeclaration(Method(HasName(StringConstructor))),
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
ArgumentCountIs(2),
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
// The first argument must have the form x.c_str() or p->c_str()
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
// where the method is string::c_str(). We can use the copy
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
// constructor of string instead (or the compiler might share
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
// the string object).
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
HasArgument(
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
0,
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
Id("call", Call(
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
Callee(Id("member", MemberExpression())),
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
Callee(Method(HasName(StringCStrMethod))),
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
On(Id("arg", Expression()))))),
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
// The second argument is the alloc object which must not be
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
// present explicitly.
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
HasArgument(
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
1,
</td></tr><tr style="background-color:rgb(229, 236, 249)"><td></td></tr><tr><td style="white-space:pre-wrap;font-family:monospace;font-size:12px;vertical-align:top">
DefaultArgument()))</td></tr></tbody></table></span><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">The next steps will be to build up better support for in-process refactorings based on the Rewriter, to build up higher-level matchers for common patterns, and to extend the low-level matcher library.</font></div>
<div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">(rietveld link: </font><a href="http://codereview.appspot.com/4552059/" target="_blank">http://codereview.appspot.com/4552059/</a>)</div>
<div><br></div><div>Cheers,</div><div>/Manuel</div>
</div><br>
</div></div></blockquote></div><br>