<div dir="ltr">Hi Zixu, I just wanted to say that this is of interest to me!<br><br>I work on a couple of FFI generation tools, and something like this would make it easier<br>for us to generate code from headers. The clang AST is pretty scary so a tool like this <div>would definitely be appreciated.<div><br>Best regards<br>Mats Larsen</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Sep 1, 2021 at 8:19 PM Zixu Wang via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;"><span>Hi All!</span><span><br></span><span><br></span><span>I’m writing to propose clang-extract-api, a tool to collect and serialize API information from header files, for example function signatures, Objective-C interfaces and protocols, and inline documentation comments. We hope it could help future tools to understand clang-based language APIs without needing to dig into the AST themselves.</span><span><br></span><span><br></span><div><span><font size="4"><b>Background<br></b></font><br></span><span><span style="font-size:14px"><b>Motivation</b></span><br><br></span><span>Library and SDK providers may find it useful to be able to create and inspect a “snapshot” of APIs they expose, for example, to check for API/ABI-breaking changes between two versions, or to automate generating documentation for the APIs. Here is a list of examples of information we want to extract from APIs:</span><span><br></span><span><div><span style="white-space:pre-wrap"> </span>• the name (spelling/mangled) of the symbol;<br></div><div><span style="white-space:pre-wrap"> </span>• the unique identifier of the symbol, for example the Unified Symbol Resolution (USR);<br></div><div><span style="white-space:pre-wrap"> </span>• the source location of the API declaration (file, line, column);<br></div><div><span style="white-space:pre-wrap"> </span>• access control of the API (public/private/protected);<br></div><div><span style="white-space:pre-wrap"> </span>• availability (available/unavailable/deprecated);<br></div><div><span style="white-space:pre-wrap"> </span>• function signatures (return/parameters);<br></div><div><span style="white-space:pre-wrap"> </span>• documentation comments attached to a symbol;<br></div><div><span style="white-space:pre-wrap"> </span>• relations with other symbols (class methods, typedef relations, struct data fields, enum constants, etc.)<br></div></span><span>Since these API information is available in the header files, which declare and distribute the APIs, we can implement a tool to extract them without invoking a compilation of the whole project to enable easy access to the information for tooling.</span></div><div><span><br></span><span><span style="font-size:14px"><b>Existing solutions</b></span><br><br></span><span>While there are some existing solutions in clang to dump symbols or AST information, they either expose unnecessary low-level details or fail to provide enough information of APIs. For example, clang -ast-dump dumps low-level details for all declarations for debug purposes and the output is not machine-parsable. Doxygen also extracts documentation comments and other information from API declarations, but its output is rendered documentation in web formats which is not flexible for other uses and tools.</span></div><div><span><br></span><span><b><font size="4">Proposal</font></b><br><br></span><span>We propose to implement this tool as a new frontend action invoked by `clang -extract-api` as show in the example below.</span></div><div><span><br></span><span><font face="Menlo">clang -extract-api \<br> header.h [more_header.h ...] or a filelist\<br> -isysroot <SDK> \<br> -target <TARGET> \<br> -I <INCLUDE PATH> \<br> -isystem <SYS INCLUDE PATH> \<br> ...<br> -o output.json</font></span></div><div><span><font face="Menlo"><br></font></span><span>It takes in the header file(s) or a filelist file containing paths to the header file(s) as the input. The header files will be parsed by clang and the extract-api action will visit the AST to extract needed information and serialize to a JSON output. Please find an example input and output attached.</span><span><br></span><span><br></span><span>The example output is based on the symbol graph format that's already used by Swift for serializing symbol information and their relations. This format can represent the required API information and is flexible and extendable as demonstrated in the example so we think it's a good starting point.</span><span><br></span><span><br></span><span>We are excited about this idea and its potential uses, and we’d love to hear feedback and suggestions!</span><br></div><div><span><br></span></div><div></div></div><div style="overflow-wrap: break-word;"><div></div></div><div style="overflow-wrap: break-word;"><div></div><div><span><br></span></div></div>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>