[cfe-dev] [GSoC] Doxygen documentation with clang

[Dmitri Gribenko]

On Wed, Mar 5, 2014 at 1:19 PM, Philipp Moeller <bootsarehax at gmail.com> wrote:
> Hello clang-devel,
>
> one of the project ideas for GSoC 2014 is a clang-based tool to
> generate documentation using doxygen-style comments in the source
> code.  I wanted to gauge the interest into such a project, see if
> someone is willing to mentor it, and provide a rough outline of what
> my idea of the project is. Any feedback on this is very welcome.

Hi Philipp,

Please excuse me for the late reply.

I am very interested in this project and I would be happy to mentor it.

Comments inline.

> 2 Prior Work
> ════════════
>
> • clang already understands doxygen-style comments to a degree and
>   attaches them to the ast:
>   [http://llvm.org/devmtg/2012-11/Gribenko_CommentParsing.pdf]
> • doxygen can already use clang as a backend
>   [http://comments.gmane.org/gmane.comp.compilers.clang.devel/29490]
> • there already is a cldoc [https://github.com/jessevdk/cldoc]
>
>
> 3 Project Plan
> ══════════════
>
> 3.1 Fully parse doxygen comments
> ────────────────────────────────
>
> Doxygen supports markdown, HTML entities, if/endif, post-definition
> documentation, file scope doc, function groups, member groups, pages,
> page hierarchies, examples, links, auto-links, and todo/bug lists (the
> dreaded xrefitem).
>
> Some of those features might seem like overkill but they usually ended
> up in doxygen because someone wanted them and they are actively used
> in "the real world" (c).
>
> The CommentParser should do its best to represent those in a useful
> fashion in the CommentAST (especially link resolving) so tools further
> down the chain can focus on their tasks only.

Link resolving is pretty important when generating self-contained
documentation files.  Clang does not attempt to resolve links right
now.  I expect that implementing this will need a significant time
investment.

Another important missing feature is attaching comments to macros.
Currently Clang can only attach comments to declarations.

> 3.2 To represent intermediately or not
> ──────────────────────────────────────
>
> The actual documentation generation tool has two options:
> • use libclang, work on the AST directly and spit out documenation.
> • let clang produce some intermediate representation (XML?) and work
>   on this
>
> The first option seems to be the easy road but would tie the
> generation directly into clang. It also seems harder to extend and
> reuse.
>
> The second option is probably the most general approach. Generating XML
> to represent the AST is actually proposed as its own GSoC project. Maybe
> it would be possible to produce a reduced XML only containing
> declarations and comments that could later be extended to feature the
> full AST. Designing this schema is probably non-trivial and should be
> well thought through.
>
> The slides on -Wdocumentation already mention the ability to produce
> XML but I couldn't figure out yet how to get that to work. From
> glancing at the schema in bindings/xml/comment-xml-schema.rng it looks
> pretty useful already, but some features (header dependencies,
> inheritance relationships) are AFAIK missing.
>
> The main benefit of an intermediate representation would be to enable
> us to build something akin to doxygen's "external projects" feature,
> which is incredibly useful (not having it would be a deal-breaker for
> some of my own projects).

Clang's XML comment representation should accurately represent the
comment in a way that is:

- extensible and future-proof,
- allows us to change the AST while maintaining backward compatibility.

Information like inheritance relationships is omitted on purpose,
because it is expected that the client is using libclang and can query
the additional information as needed.

> 3.3.3 Database + Web-server
> ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
>
> A special case for HTML. Provide a database and a web-frontend that
> can be hosted. Seems interesting for fast search functions and live
> documentation updates. clang-server where are you?

This looks like a very promising approach that does not just provide
the same functionality as Doxygen does, but introduces new value.
This can actually become the foundation for the clang-server itself!
The basic functionality for live updates -- tracking dependencies
between source files, indexing and reindexing will be useful for both
documentation server and clang-server.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr at gmail.com>*/