[cfe-dev] Design: clang-format

[Douglas Gregor]

Hi Manuel,

On May 11, 2012, at 9:27 AM, Manuel Klimek wrote:

> Hi,
> 
> we're working on the design of clang-format; there are quite a few open questions, but I'd rather get some early feedback to see whether we're completely off track somewhere.

Great!

> The main doc is on google docs:
> https://docs.google.com/document/d/1gpckL2U_6QuU9YW2L1ABsc4Fcogn5UngKk7fE5dDOoA/edit
> 
> For those of you who prefer good old email, here is a copy of the current state. Feel free to add comments in either.
> Design: clang-format
> 
> This document contains a design proposal for a clang-format tool, which allows C++ developers to automatically format their code independently of their development environment.
> Context
> 
> While many other languages have auto-formatters available, C++ is still lacking a tool that fits the needs of the majority of C++ programmers. Note that when we talk about formatting as part of this document, we mean both the problem of indentation (which has been largely solved independently by regexp-based implementations in editors / IDEs) and line breaking, which proves to be a harder problem.
> 
> There are multiple challenges to formatting C++ code:
> a vast number of different coding styles has evolved over time
> many projects value consistency over conformance and dislike style-only changes, thus making it important to be able to work with code that is not written according to the most current style guide
> macros need to be handled properly
> it should be possible to format code that is not yet syntactically correct
> Goals
> 
> Format a whole file according to a configuration
> Format a part of a file according to a configuration
> Format a part of a file while being consistent as best as possible with the rest of the file, while falling back to a configuration for options that cannot be deduced from the current file
> Integrating with editors so that you can just type away until you’re far past the column limit, and then hit a key and have the editor layout the code for you, including placing the right line breaks
> Non-goals
> 
> Indenting code while you type; this is a much simpler problem, but has even stronger performance requirements - the current editors should be good enough, and we’ll allow new workflows that don’t ever require the user to break lines
> The only lexical elements clang-format should touch are: whitespaces, string-literals and comments. Any other changes ranging from ordering includes to removing superfluous paranthesis are not in the scope of this tool.
> Per-file configuration: be able to annotate a file with a style which it adheres to (?)
> Code location
> 
> Clang-format is a very basic tool, so it might warrant living in clang mainstream. On the other hand it would also fit nicely with other clang refactoring tools. TODO: Where do we want clang-format to live?

I think clang-format should live with the refactoring tools, wherever they go. However, refactoring is going to be a crucial technology for Clang going forward, which almost certainly means that it should migrate into mainline Clang at some point.

>  Parsing approach
> 
> The key consideration is whether clang-format can be based purely on a lexer, or whether it needs type information, and we need the full AST. 
> 
> We believe that we will need the full AST information to correctly indent code, break lines, and fix whitespace within a line.
> 
> Examples:
> 
> AST-dependent indentation:
> callFunction(foo<something,
>                  ^ line up here, if foo is a template name
>              ^ line up here otherwise
> 
> AST-dependent line breaking:
> Detecting that ‘*’ is an binary operator in this case requires parsing; if it is a binary operator, we want to line-break after it, if it is a unary operator, we want to prevent line breaking
> 
> result = variable1 * variable2;
> 
> AST-dependent whitespace inside lines:
> a * b;
>    ^ Binary operator or pointer declaration?
> a & f();
>    ^ Binary operator or function declaration?

I wonder how well we can do simply with the lexer and preprocessor. Introducing the AST traversal adds a lot of complication, but you're right that it's necessary to do a great job.

> Challenge: Preprocessor
> Not every line in a program is covered by the AST - for example, there are unused macro definitions, various preprocessor directives, #ifdef’ed out code, etc.
> 
> We will at least need some form of lexing approach for the parts of a source file that cannot be correctly indented / line broken by looking at the AST.
> 
> Algorithm
> 
> Visit all nodes on the AST; for each node that is part of a macro expansion, consider all locations taking part in that macro expansion. If the location is within the range that need to be indented, look at the code at the location, the rules around the node, and adjust whitespace as necessary. If the node starts a line, adjust the indent; if a node overflows the line, break the line. TODO: figure out what to do with the lines that are not visited that way.
> Configuration
> 
> To support a majority of developers, being able to configure the desired style is key. We propose using a YAML configuration file, as there’s already a YAML parser readily available in LLVM. Proposals for more specific ideas welcome.

Seems reasonable.
>  Style deduction
> 
> When changing the format of code that does not conform to a given style configuration, we will optionally try to deduce style options from the file first, and fall back to the configured layout when there was no clear style deducible from the context.
> TODO: Detailed design ideas.

Yes, please!
> Interface
> 
> This is a strawman. Please shoot down.
> 
> Command line interface:
> Command line interfaces allow easy integration with existing tools and editors.
> 
> USAGE: clang-format <build-path> <source> [<column0> <line0> <length0> [...]] [-- list of command line arguments to the parser]
> 
> <columnN> <lineN> <lengthN>: Specifies a code range to be reformatted; if no code range is given, assume the whole file.

For an editor to use this functionality efficiently, we'll want it to go into a shared library (e.g., libclang).

I'm *thrilled* that you're looking into this!

	- Doug

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20120514/4185aeaf/attachment.html>