[cfe-dev] RFC: clang-doc proposal

[Jonathan Müller via cfe-dev]

On Dec 4, 2017 9:21 PM, "Julie Hockett via cfe-dev" <cfe-dev at lists.llvm.org>
wrote:

This proposal is to build a new Clang tool for generating C/C++
documentation, similar to other modern documentation generators such as
rustdoc.  This tool would be a modular and extensible documentation
generator for C/C++ code. It also would introduce a more elegant way of
documenting C/C++ code, as well as maintaining backwards-compatibility with
Doxygen-style markup to generate documentation.

Today, Doxygen is a de-facto standard for generating C/C++ documentation.
While widely used, the tool itself is a bit cumbersome, its output is both
aesthetically and functionally lacking, and the non-permissive license
combined with outdated codebase make any improvements difficult. This new
tool would aim to simplify the overhead of generating documentation,
integrating it into a Clang tool as well as allowing existing comments to
continue to be used. It would also allow for relatively easy adaptation to
new language features, as it would be built on the Clang parser and would
use the Clang AST to generate documentation.



I'm currently working on standardese, which aims to do exactly that:
github.com/foonathan/standardese


Proposed Tool

The proposed tool would consist of two parts. First, it would have a
frontend that consumes the Clang-generated AST and generates an
intermediate representation of the code and documentation structure,
including additional Markdown files. Second, it would have a set of backend
modules that consume that representation and output documentation in a
particular format (e.g. Markdown, HTML/website, etc.).


It is designed in a similar way, but the intermediate format is just
in-memory.

The frontend would be a new tool that uses the Clang parser, which can
already parse C/C++ documentation comments (using -Wdocumentation option).
It can be easily used through the LibTooling interface, similarly to other
Clang tools such as clang-check or clang-format. The initial steps in this
project would be to build this tool using Clang's documentation parser.
This tool would be able to attach comments to both functions, types, and
macros and resolve declaration references, both of which will be useful in
generating effective documentation. Since a good deal of existing C/C++
code uses the Doxygen documentation comment style, which is also supported
by Clang's parser (and Doxygen itself can use Clang to parse these comments),
this is the syntax we are going to support as well. In the future, we would
also like to support Markdown-style comments, akin to Apple Swift Markup.

standardese uses libclang (which was a mistake in hindsight), and has its
own comment parser as it also supports markup and special commands.

For implementation, this tool will use the JSON Compilation Database format
to integrate with existing build systems. It would also have subcommands to
choose which parts of the code will be documented (e.g. all code, all
public signatures, all comment-documented signatures). Once the code is
processed, the tool will write out the internal representation of the the
documentation in an intermediate representation, encapsulating the
necessary information about the code, comments, and structure. This will
allow backend tools to take the output and transform it as necessary.

This works as described, but the intermediate representation isn't written
out.

The backend modules would cover different possible outputs for the defined
intermediate representation. Each module will consume the representation
and output documentation in a specific format. Initially, we propose to
focus on a module that generates Markdown files, in order to make the first
version as simple as possible. Markdown files are automatically rendered on
a number of sites and systems, as well as being clear and uncluttered in
raw text form. It is also relatively easy to convert Markdown files into
other formats, making it a good starting target. An additional module would
target HTML/website output.


Multiple backends are supported (the new version on develop currently only
has XML and HTML though)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20171205/0c8fca19/attachment.html>