[cfe-dev] Criteria for Accepting Language Extensions into Clang

[Douglas Gregor]

On Jul 25, 2011, at 1:23 PM, George Russell wrote:

> Hi,
> 
> I have implemented a small language extension to C++, which motivates my 
> interest in Clang + language extensions 
> (https://bitbucket.org/grrussel/constcpp/wiki/Home) - which I would like 
> to make available to a wider user base (if there is, in fact, interest 
> in it) without being required to myself replicate the Clang / LLVM 
> projects packaging work.
> 
> On 21/07/2011 19:40, Douglas Gregor wrote:
>> Clang has always been designed as a platform for experimentation, and my hope is that Clang will be the premier vehicle for innovation in the C family of languages, establishing existing practice for the various standards committees. Sooner or later, the successful experiments will become proposed extensions for Clang, at which point we either fully commit to the extension--adding it into the tree and supporting it for many years ahead--or we must decide that the extension is not suitable for inclusion in Clang at this time. To aid in that decision, I propose the following seven criteria:
>> 
> I wonder if there is any potential for what I have seen in LLVM - e.g. 
> back-ends that are marked as unsupported, experimental; or of features / 
> extensions being dropped if there is not an active maintainer.

This works extremely well when the code is isolated. In LLVM land, for example, we can easily just delete an unmaintained back-end. Moreover, it's far simpler to communicate this to users: "there is no more Itanium back end as of version x.y" vs. "your code may break if compiled with version x.y if you happened to rely on this extension."

> In other 
> words, does inclusion necessarily require the unambiguous commitment to 
> future support?

Yes, it does require a commitment for long-term support. We maintain a *lot* of baggage we've inherited from other compilers' extensions (including some truly horrible, half-baked ones), and we can't afford to make the situation works.

> Is there any potential for features explicitly marked as 
> experimental and with the potential for future changes (including loss 
> of backwards compatibility) and which could, at the discretion of core 
> developers, simply be dropped between releases?

The problem with experimental features is that users come to rely on them (often accidentally), so removing them later on becomes very, very tricky. 

>> 1) Evidence of a significant user community: This is based on a number of factors, including an actual, existing user community, the perceived likelihood that users would adopt such a feature if it were available, and any "trickle-down" effects that come from, e.g., a library adopting the feature and providing benefits to its users.
> 
> One motivation for getting an extension into Clang / LLVM is the ability 
> to attract a larger community of users - it would be very easy to say, 
> if you want to try "language extension X" it is available in some given 
> Clang release for the range of platforms and targets it supports, as 
> opposed to saying a) build it yourself, to a potential user or b) 
> building + distributing the equivalent to a Clang / LLVM release yourself.

Yes, there is a well-known chicken-and-egg problem here. How to attract a larger user base, if the feature isn't in a compiler they have? But how can we commit to putting a feature in the compiler, if there is no user base? I don't have a good answer for this, except to hope that truly compelling features will get people excited enough to patch their Clang and get coding.

>> 4) Representation within the appropriate governing organization: For extensions to a language governed by a standards committee (C, C++, OpenCL), the extension itself must have an active proposal and proponent within that committee and have a reasonable chance of acceptance. Clang should drive the standard, not diverge from it.
> Is there any potential for an extension that is not intended to become a 
> feature of language X version current + 1? I am aware of several 
> extensions to various languages not intended to be such.

In my opinion, no. A new C or C++ feature is only truly useful if it's on track to be standardized by the committee. A Clang-only C or C++ feature serves only a small (but growing!) slice of the C/C++ user base, and, worse, may eventually end up conflicting with features added by the appropriate standardization committee. 

>> 6) A high-quality implementation: Naturally, the implementation must fit well into Clang's architecture, follow LLVM's coding conventions, and meet Clang's quality standards, including high-quality diagnostics and rich AST representations. This is particularly important for language extensions, because users will learn how those extensions work through the behavior of the compiler.
>> 
> Yes; I would not argue that my extension is of such a quality, being at 
> present a proof of concept; Is there scope for an incremental process 
> where implemented extensions that the implementer desires to be merged 
> could be reviewed, to allow feedback from the core developers of Clang 
> etc to increase the quality of the implementation?


You can always solicit feedback on the mailing lists, but it's no sure thing: if someone is interested in your extension enough to dig into the implementation, you could get some great feedback. Core developers will weigh in if they can.

We have talked about the general issue that it's fairly hard to get any real usage experience for a feature until it's inside the Clang tree, yet we can't add a feature into the Clang tree until it's proven itself. One idea that comes up from time to time is to have an experimental Clang branch where the rules are greatly relaxed: people can add features and incrementally improve them. If they prove themselves, we can consider merging the implementation over to the main Clang branch. I would consider having such a branch, but I personally do not have the time to maintain it. Someone else would need to handle period (daily?) merges from trunk to that branch, keep the compiler usable, etc.

	- Doug