<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On May 17, 2017, at 12:35 PM, Dean Michael Berris via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" class="">cfe-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class=""><br class="Apple-interchange-newline">On 18 May 2017, at 01:29, Gábor Horváth via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" class="">cfe-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class=""><div class=""><div class=""><div class="">Hi!<br class=""><br class=""></div>I wanted to revive the discussion on adding API notes feature to Clang.<span class="Apple-converted-space"> </span><br class=""><br class=""></div></div><div class=""><b class="">The description of the feature from the original discussion</b><br class=""><br class="">"API notes solve a not-uncommon problem: we invent some new Clang attribute that would be beneficial to add to some declarations in system headers (e.g., adding a ‘noreturn’ attribute to the C ‘exit’ function), but we can’t go around and fix all of the system headers everywhere. With API notes, we can write a separate YAML file that states that we want to add ‘noreturn’ to the ‘exit’ function: when we feed that YAML file into Clang as part of normal compilation (via a command-line option), Clang will add ‘noreturn’ to the ‘exit’ function when it parses the declaration of ‘exit’."</div><div class=""><br class="">The old discussion can be found here:<span class="Apple-converted-space"> </span><a href="http://lists.llvm.org/pipermail/cfe-dev/2015-December/046335.html" class="">http://lists.llvm.org/pipermail/cfe-dev/2015-December/046335.html</a><br class=""><br class=""></div><b class="">The summary of the discussion</b><br class=""><br class=""></div><div class="">* It would be useful for the static analyzer to attach additional info to the functions<br class="">* There were already other trials to get similar feature working, see<span class="Apple-converted-space"> </span><a href="https://reviews.llvm.org/D13731" class="">https://reviews.llvm.org/D13731</a><br class=""></div><div class="">* Anna has a nice summary what is the problem with augmented declarations:<span class="Apple-converted-space"> </span><a href="http://lists.llvm.org/pipermail/cfe-dev/2015-December/046378.html" class="">http://lists.llvm.org/pipermail/cfe-dev/2015-December/046378.html</a><br class=""></div><div class="">* C++ support is requested by the community, this is missing right now.<span class="Apple-converted-space"> </span><br class=""></div><div class="">* Wider range of annotation support is missing. It is also requested by the community.<span class="Apple-converted-space"> </span><br class=""></div><div class="">* Parameter annotations are supported<br class=""></div><div class="">* One of the concerns is the performance<br class=""><br class=""></div><div class=""><b class="">The case for adding API Notes</b><br class=""><br class=""></div><div class="">* Importing annotations from external source looks to be an interesting feature for the community<br class=""></div><div class="">* Sean presented some API checkers which works based on special annotations:<span class="Apple-converted-space"> </span><a href="http://llvm.org/devmtg/2017-03//2017/02/20/accepted-sessions.html#29" class="">http://llvm.org/devmtg/2017-03//2017/02/20/accepted-sessions.html#29</a><br class=""></div><div class="">* Similar feature can be used to provide fake implementations for functions. It could help to finish a half done feature of the Static Analyzer that was implemented during a GSoC project:<span class="Apple-converted-space"> </span><a href="https://www.google-melange.com/archive/gsoc/2014/orgs/llvm/projects/xazax.html" class="">https://www.google-melange.com/archive/gsoc/2014/orgs/llvm/projects/xazax.html</a><br class="">* Performance might be crucial for regular compilation, but less of a problem for the Clang Static Analyzer which tends to be slower than compilations, so it is less likely to be bottlenecked by this phase<br class=""></div><div class="">* It would also be possible to import sanitizer/optimizer/codegen related annotations<br class=""></div><div class=""><br class=""></div><div class=""><b class="">Proposed roadmap</b><br class=""></div><div class=""><br class=""></div><div class="">* Commit the feature as is<br class=""></div><div class="">* Extend it with C++ support (namespaces, overloading, templates..)<br class=""></div><div class="">* Extend it with additional annotations, and attaching custom data (like fake function bodies)<br class=""></div><div class=""><br class=""></div><div class="">What do you think? What are the main concerns with this feature?<br class=""><br class=""></div></div></div></blockquote><div class=""><br class=""></div><div class="">I'm interested in having something like API notes available in Clang, definitely. I have a few questions though:</div><div class=""><br class=""></div><div class="">- Is YAML absolutely necessary?</div></div></div></blockquote><div><br class=""></div><div>It’s not necessarily, but it’s convenience. Swift relies heavily on the YAML form; it’s been deployed for quite a while, so if API notes eventually take some other form, we’ll still need a translation from YAML to that new form for backward compatibility.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class=""> What I think I don't understand is what the actual data structure is that it will represent. </div></div></div></blockquote><div><br class=""></div><div>Here’s an example of API notes as used for Swift:</div><div><br class=""></div><div><span class="Apple-tab-span" style="white-space:pre"> </span><a href="https://github.com/apple/swift/blob/master/apinotes/Foundation.apinotes" class="">https://github.com/apple/swift/blob/master/apinotes/Foundation.apinotes</a></div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="">Is it a configuration for something that will actually imbue the attributes into the IR? </div></div></div></blockquote><div><br class=""></div><div>The entries in API notes are typically mapped to attributes in the Clang *AST* at the point where the corresponding entity is parsed. Some will affect warnings, some will affect the generated IR, etc.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="">Or is it a dump of an in-memory data structure, that specifies a side-table of attributes for the generated IR?</div></div></div></blockquote><div><br class=""></div><div>It’s not an in-memory data structure. It’s just a format that can describe declarations and apply additional information to them.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="">- What is the overlap with the sanitiser special case list, which provides regular expressions and a simple configuration language? Can one be ported to be built on top of the other? Is there a roadmap/plan for unifying that features?</div></div></div></blockquote><div><br class=""></div><div>The sanitizer/analyzer special case would involve adding more fields into the YAML, which then map to whatever internal data structure the sanitizer/analyzer needs.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="">- What is the programmatic API for imbuing the attributes before IR generation? Say I'm using clang as a library, is there a data structure (in-memory) that I can programmatically define so that the attributes are applied to the IR as a transformation? If so, should this be part of the LLVM back-end instead? Or should it be a stand-alone library that clang uses in the static analysis phase? I can certainly imagine this being useful in LTO or PGO, not necessarily tied to Clang.</div></div></div></blockquote><br class=""></div><div>Clang’s Sema is responsible for mapping API notes to the AST; see <a href="https://github.com/apple/swift-clang/blob/stable/lib/Sema/SemaAPINotes.cpp" class="">https://github.com/apple/swift-clang/blob/stable/lib/Sema/SemaAPINotes.cpp</a> for the actual implementation. This is the Right Answer, because it adds attributes at the same point they would get added if the annotations were in the source code itself, so all downstream clients see the effects of the attributes.</div><div><br class=""></div><div>To make it more custom, what you really need is a generalized attribute mechanism in Clang itself that makes it easy to, e.g., add your own attribute (programmatically?), map those attributes down to IR attributes, and have API notes be general enough to support those attributes as well.</div><div><br class=""></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>- Doug</div><div><br class=""></div><br class=""></body></html>