<div dir="ltr"><div>Your comment is very surprising to me. <br></div><div><div>The points you make sound to me like advantages not disadvantages:<br><br></div></div><div>> <span style="font-size:12.8px">C++ AST can be created that is both useful and not overly specific to a single compiler.</span></div><div><span style="font-size:12.8px">Maybe, but what more important is an AST you can build, analyse and transform easily.</span></div><div><br></div><div>> <span style="font-size:12.8px">The IPR does not handle macros</span><span style="font-size:12.8px"> [...]</span></div><div><span style="font-size:12.8px">Why should it ? - what would it be good for, besides having source-locations ? - how is this currently handled in clang ?</span></div><div><span style="font-size:12.8px"> As far as i know clang's AST has no notion of macros.</span><br></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><br></span></div><div><div>I imagine it could be very handy to have multiple ASTs:</div><div>One having only nodes for Preprocessor (ppAST) constructs and one for C++ (cppAST).</div><div>The Preprocessor could process ppAST and return cppAST alongside with a sourcemap.</div><div>We could even go a step further and have separate ASTs for C++ with and without templates.</div></div><div>This would make the codebase more functional and composable.</div><div><br></div><div>To get a proper source-location we then just need to follow the sourcemaps.</div><div><br></div><div>Dealing with real (immutable) ASTs would make (de-)serialization, cloning and comparing an easy task.</div><div>As far as i can see, the preprocessor is a big liability:</div><div>The preprocessor is stateful and macros can transform the sourcefile in almost any imaginable way.</div><div><br></div><div>As far as i can see the c++-parser could be paralyzed. </div><div>However this is only possible if the parser is decoupled from the preprocessor.</div><div><br></div><div><br></div><div><div style="font-size:12.8px">> - does not mimic C++ language irregularities; general rules are used, rather than long lists of special cases</div><div style="font-size:12.8px">We still can write an validator to be sure the AST conforms to a specific c++-standard.</div><div style="font-size:12.8px">I think that having a nice and regular AST would simplify working with the AST. </div><div style="font-size:12.8px">Writing visitors and pattern matchers for semantic analysis might become easier. </div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">> <span style="font-size:12.8px">Generally, I would not trust a representation that I can't generate code from to be correct enough for tools.</span></div><div style="font-size:12.8px">Well, my goal would definitely be to generate code from the AST. If information are missing in the representation then we need to improve it.</div><div><span style="font-size:12.8px">IMHO the main message of the paper is that there is that you can build an AST representation that is more regular and still complete.</span></div><div><br></div></div><div><br></div><div>> <span style="font-size:12.8px">- Unfortunately, a program cannot fully automate the generation of “skeletons.” If our aim is portability, we still need to (by hand) eliminate non-standard additions to the contents of header file.</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">can you elaborate? - what do you mean by "skeletons" and to which header files are you referring to?</span></div><div><span style="font-size:12.8px"><br></span></div><div class="gmail_extra"><div class="gmail_quote">2017-02-06 15:45 GMT+00:00 Manuel Klimek <span dir="ltr"><<a href="mailto:klimek@google.com" target="_blank">klimek@google.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I haven't looked too deeply into it, but from talking to various clang developers, the common theme is disbelieve that a high-level C++ AST can be created that is both useful and not overly specific to a single compiler.<div><br></div><div>From the paper referenced in the project, I see multiple things that make it seem not interesting to me from a point of refactoring and semantic analysis:</div><div>- The IPR does not handle macros before their expansion in the preprocessor.<br></div><div>- does not mimic C++ language irregularities; general rules are used, rather than long lists of special cases</div><div>- Unfortunately, a program cannot fully automate the generation of “skeletons.” If our aim is portability, we still need to (by hand) eliminate non-standard additions to the contents of header file.</div><div><br></div><div>Generally, I would not trust a representation that I can't generate code from to be correct enough for tools.</div></div><div class="gmail-m_-3548957017129714506HOEnZb"><div class="gmail-m_-3548957017129714506h5"><br><div class="gmail_quote"><div dir="ltr">On Wed, Feb 1, 2017 at 8:32 PM Reid Kleckner via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">Yes, the clang "abstract syntax tree" is often jokingly referred to as the "concrete syntax graph". We try to provide a generally useful representation, but being a fast production C++ compiler comes first. Clang's AST is very concrete. You have to know a lot about it to navigate it. There is no "Node" base class that you can use as a cursor to navigate around Decls, Exprs, Types, and TemplateArguments. Template instantiation, the closest thing I can think of to cloning, is done relatively manually with TreeTransform.<div class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"><br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"></div><div class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">I think it would be nice to revisit the design of clang's AST to simplify it, normalize it, and abstract it, but it is not a task to be taken lightly, and I don't expect it to happen in the near future.</div></div><div class="gmail_extra gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"><br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"><div class="gmail_quote gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">On Wed, Feb 1, 2017 at 8:44 AM, Gaetano Checinski via cfe-dev <span dir="ltr" class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"><<a href="mailto:cfe-dev@lists.llvm.org" class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg" target="_blank">cfe-dev@lists.llvm.org</a>></span> wrote:<br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"><blockquote class="gmail_quote gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">As the AST is not really a Tree as it seems to have circular references, working with the AST is sometimes a bit messy (eg. cloning).<div class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"><br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"></div><div class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">A while ago Stroustroup pointed me to Gabriel Dos Reis' work on a different approach to represent C++-AST: <a href="https://mailtrack.io/trace/link/a5c184fad2cf94fbf0449fa233263072ebc34ea8?url=https%3A%2F%2Fgithub.com%2FGabrielDosReis%2Fipr&signature=9a19f2a0a41d5ceb" class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg" target="_blank">https://github.com/GabrielDosR<wbr>eis/ipr</a></div><div class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"><br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"></div><div class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">Did anyone try to integrate his work into clang or has an opinion to share ?<br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"></div><img width="0" height="0" class="gmail-m_-3548957017129714506m_-5452629544365183520m_-1034651269622219800m_-8147391716180187190mailtrack-img gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"></div>
<br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">______________________________<wbr>_________________<br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">
cfe-dev mailing list<br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">
<a href="mailto:cfe-dev@lists.llvm.org" class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg" target="_blank">cfe-dev@lists.llvm.org</a><br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">
<br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"></blockquote></div><br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg"></div>
______________________________<wbr>_________________<br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">
cfe-dev mailing list<br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">
<a href="mailto:cfe-dev@lists.llvm.org" class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg" target="_blank">cfe-dev@lists.llvm.org</a><br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br class="gmail-m_-3548957017129714506m_-5452629544365183520gmail_msg">
</blockquote></div>
</div></div></blockquote></div><br></div><span class="gmail-mt-tool-email-notification-true"></span><span class="gmail-mt-tool-email-tracking-true"></span><br><br><br><div class="mt-signature"><a href="https://mailtrack.io/trace/link/29fe4deb755f2a860d95181bcf80131f16c45e93?url=https%3A%2F%2Fmailtrack.io%2F&signature=d1df2a3c7469f8d7" class="gmail-mt-signature-logo" style="text-decoration:none"><img src="https://s3-eu-west-1.amazonaws.com/mailtrack-crx/icon-signature.png" width="16" height="14"> </a><font color="#999999" class="mt-signature-text">Sent with <a href="https://mailtrack.io/install?source=signature&lang=en&referral=gaetano.checinski@gmail.com&idSignature=22" class="mt-install">Mailtrack</a></font></div><img width="0" height="0" class="mailtrack-img" src="https://mailtrack.io/trace/mail/8ba1b8824c012f875bf1a67de8774b902460c2d5.png?u=931501"></div>