[cfe-dev] RFC: proposing to relax standardization requirements for Clang extensions

[Aaron Ballman via cfe-dev]

On Wed, Dec 8, 2021 at 3:48 PM Richard Smith <richard at metafoo.co.uk> wrote:
>
> On Tue, 7 Dec 2021 at 13:22, Aaron Ballman via cfe-commits <cfe-commits at lists.llvm.org> wrote:
>>
>> tl;dr: our Clang "get involved" page implies that proposed extensions
>> to Clang must also be proposed to a standards committee
>> (https://clang.llvm.org/get_involved.html#criteria). This is a good
>> goal, but is not inclusive or something we enforce with any
>> consistency. I'm proposing to clarify our goals in this space and to
>> better reflect existing practice.
>>
>> Inclusivity Issues:
>> Participating in a standards body is not always free. In fact, it can
>> be quite expensive depending on where you live and which committee you
>> want to join. People who wish to join the C or C++ standards committee
>> as a US national body member have yearly dues in excess of $2000/yr.
>> Each national body sets their membership fees individually and the
>> prices range anywhere from free to $7500+/yr. Also, a standards
>> proposal is not a "one and done" activity and often takes multiple
>> meetings to see even a trivial proposal adopted, but may require
>> multiple years for larger proposals. For example, the [[]] attributes
>> proposal took three years, and _BitInt took over two years with
>> multiple senior-level engineers working on it. Further, there are
>> associated travel costs (in non-pandemic times, anyway) with attending
>> meetings. This is a very significant financial and time burden that I
>> don't think we should require community members to take on.
>>
>> Consistency Issues:
>> We have never been consistent at enforcing this requirement. Here are
>> some examples off the top of my head:
>>
>> * Nullability qualifiers -- never proposed to WG14
>> * VLAs in C++ -- never proposed to WG21
>> * [[]] attributes in C -- proposed to WG14, accepted
>> * _ExtInt -- proposed to WG14, accepted as _BitInt
>> * enums with a fixed underlying type in C -- proposed to WG14 (not by
>> a Clang community member), still not accepted yet
>> * (This list is not intended to be exhaustive.)
>
>
> VLAs in C++, [[]] in C, and enums with a fixed underlying type are all instances of accepting a feature from one language mode in another. For those cases, we do have a relevant standard on which to base our behaviour, so I think those meet the current policy.

I am less convinced of that, but understand where you're coming from.
VLAs in C++ are a great example -- the C standard says what VLAs mean,
but that doesn't cover very important topics like what happens with a
VLA in a constant expression in C++. And the presence of our extension
in this space can constrain WG21 from making future changes in the
area because of fear of breaking our users of the extension.

> Nullability qualifiers are part of the Mac OS language platform's SDK. In this instance, Apple as a platform vendor decided to add an extension, and we as vendors of a compiler that intends to support that platform, implement that extension. I don't think this is all that different from our supporting, say, <immintrin.h> to support intel platforms. Or our supporting MS extensions to support the MS platform.

And yet, they impact standardization nonetheless. There was a recent
RFC about nonnull attributes that someone wishes to propose to WG14
and the nullability attributes were a part of that discussion. If a
platform vendor needs to add an extension and that extension is
sufficiently general enough to be standardized, I read the existing
documentation as requiring them to bring it to a standards body.
(Also, it seems to me that platform vendors are the one entity in the
community that generally has the resources necessary for successful
standardization -- it strikes me as odd to make it easier to exempt
them from a requirement here.)

That said, I definitely take your point that there are extensions
vendors need to add for platform support which are not appropriate for
standardization. (e.g., adding new intrinsics to support the platform,
hooks required by the platform, attributes, etc.)

> For _ExtInt, I specifically asked that this be proposed to WG14, and it was (https://reviews.llvm.org/D59105#1422089).
>
> So, I think we may be missing some detail from the policy:
> -- sometimes the relevant organization may be the owner of a platform or architecture we care about supporting, rather than a language

+1

> -- features supported for one language mode or platform are allowed to be supported in other configurations too; if we're paying the cost of supporting a feature, we should get as much benefit from it as we can

I agree with this up to a point. I definitely agree we want to stretch
our value where we can, but I would argue that support in one language
mode is insufficient for automatic inclusion as an extension in
another language mode unless it has a written specification for the
behavior in that other language mode. For example, we don't document
VLA support in C++ in any meaningful way
(https://clang.llvm.org/compatibility.html#vla are the only docs I
could find and that's far from a language specification), and I think
that's a situation we should avoid, but if we documented it to a level
that could be reasonable for a standards committee to consider as a
proposal, that would be sufficient to alleviate my concerns.

Actually, a somewhat separate RFC I should probably write is: we
really should have explicit documentation for all our
implementation-defined behavior, which includes extensions, and we
should be requiring this as part of the code review process.

>> Coupled with the time and monetary costs associated with
>> standardization, inconsistently applying something we say "must"
>> happen in our documentation is ripe for potential abuse (both
>> malicious and unintentional).
>>
>> To this end, I'm proposing a change along the lines of:
>>
>> -  <li>Representation within the appropriate governing organization: For
>> -  extensions to a language governed by a standards committee (C, C++, OpenCL),
>> -  the extension itself must have an active proposal and proponent within that
>> -  committee and have a reasonable chance of acceptance. Clang should drive the
>> -  standard, not diverge from it. This criterion does not apply to all
>> -  extensions, since some extensions fall outside of the realm of the standards
>> -  bodies.</li>
>> +  <li>Plausibility of standardization where applicable: Extensions with an
>> +  active proposal within a standards committee (C, C++, OpenCL, etc.) are
>> +  preferred when appropriate. Proposed Clang-specific extensions that are being
>> +  considered by a standards committee must have a feedback loop between the
>> +  community and the committee. Clang should drive the standard, not diverge
>> +  from it, so proposals currently in a standardization pipeline should not be
>> +  treated as finalized features in Clang until the standardization process has
>> +  completed and review can be done against the standard defining the feature.
>> +  Regardless of whether an extension is proposed for standardization, it must
>> +  be a conforming extension and it should not knowingly impede future
>> +  standardization efforts.
>> +  </li>
>>
>> (I'll post an actual review for the changes, but I wanted folks to see
>> what I was going for as part of the RFC.)
>>
>> The goal of the change is to make the following clear:
>>
>> * We /prefer/ extensions to go through a standards body whenever possible
>> * We require extensions to be conforming and not knowingly impede
>> future standardization efforts
>> * Not all extensions are appropriate for standardization and that's fine
>> * If an extension is proposed to a standards committee, we expect the
>> community to have an active feedback loop with the committee
>> * Once a feature has been standardized, we expect Clang to expose the
>> standardized feature in a conforming way
>>
>> I am wondering if others in the community feel we should make
>> adjustments along these lines to our getting involved page?
>
>
> I'm hesitant about this direction. Every extension we carry imposes a burden on the project as a whole; while we ask people to stick around and maintain extensions they care about, that only really works in the short term, and in any case our existing set of extensions impose a combinatorial cost on future extensions. Limiting ourselves to extensions we need to support for compatibility with current and future standards helps to reduce this complexity.

That's a fair point. I think we're in alignment on wanting to limit
extensions and this relaxation makes it sound like we're interested in
the opposite outcome.

> Also, I think it's important that for every extension we carry that could and should be standardized, we encourage the folks proposing it to pursue standardization.

I agree with the spirit of this.

> I agree that getting things standardized is hard and can be expensive, and that does suck, but if the alternative is saying that we'll carry a grab-bag of extensions that no-one has done the work to standardize, that will be worse.

I contend that's the status quo we're in today, but that's somewhat
orthogonal to my concern. We have a lot of corporate backing in the
community from companies that participate on standards bodies, and I
think the current requirements largely make sense for those
contributors. I think the current requirements are too high of a
burden for people who are not backed by a corporation willing to fund
their standards efforts, and given that we aim to be inclusive, we
shouldn't exclude contributors based on financial burdens.

"that does suck" carries a lot of weight in that sentence. The
standardization efforts for [[]] started from an existing
specification in C++ and took over three years and required a single
senior-level engineer to attend eleven standards meetings (about half
of which required international travel). The standardization efforts
for _ExtInt took over two years and required four senior-level
engineers going to eight standards meetings *and is still underway*.
The travel expenses for this one were lessened due to WG14 switching
to a fully virtual format during the pandemic. I do not have an exact
financial figure I could share, but I would say it's reasonable to
assume that the cost of each of these features individually is more
than would be reasonable to expect an individual to bear.

> We've seen the effects of GNU C and GNU C++, and their collection of unstandardized extensions, and this rule is an attempt to avoid creating a significantly different "Clang C" or "Clang C++" dialect.

Agreed that we want to avoid making a Clang dialect.

> That said: I think there is room to relax our requirements here for some specific carved-out areas: at least when adding attributes and builtin functions, I don't think we need to look for representation in a standards body.

Strong +1 to these being extensions that we've effectively never
enforced our requirements for.

I think we may be somewhat aligned in our goals, but my suggested
wording may not make that sufficiently clear. Do you think there's
enough here to be worth me putting up a code review that we can try to
wordsmith into better shape, or would you prefer to discuss in this
thread as part of the RFC until we have more community input?

Thanks!

~Aaron

>
>>
>> Thanks!
>>
>> ~Aaron
>> _______________________________________________
>> cfe-commits mailing list
>> cfe-commits at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits