[cfe-dev] RFC: Nullability qualifiers

[b17 c0de]

Hi Doug,
Thanks for your changes. I think the choice you made makes the most sense.

Can you look at the nullability-completeness bug with templated types? Or
should I make a bug report?

On Mon, Jun 29, 2015 at 8:12 PM, Douglas Gregor <dgregor at apple.com> wrote:

>
> On Jun 29, 2015, at 10:57 AM, Douglas Gregor <dgregor at apple.com> wrote:
>
>
> On Jun 27, 2015, at 8:03 AM, b17 c0de <b17c0de at gmail.com> wrote:
>
> Just to clarify...
>
> __has_extension(nullability) always returning true is exactly what I would
> expect from reading the clang documentation. I would also expect that
> __has_extension(assume_nonnull) always returns true (which it currently
> doesn't).
>
>
> __has_extension(assume_nonnull) should always be true; fixed in r240969.
>
> I am hoping someone can explain why GNU mode is required for
> __has_feature(nullability) and __has_feature(assume_nonnull) to return
> true.
>
>
> GNU mode was meant to capture “the default behavior”, and gets disabled
> when one specifies stricter conformance to the language standard. Rather,
> nullability is an extension that is always available, and only a
> “standardized” language feature in Objective-C (where Clang is essentially
> the standard). In retrospect, we shouldn’t straddle the fence this way with
> GNUMode: either it’s an always-on feature (__has_feature and
> __has_extension are always true for both) or it’s only a language feature
> in Objective-C (__has_feature is true only in Objective-C mode,
> __has_extension is always true). I tend to prefer the former.
>
>
> r240976 makes these always-true.
>
> - Doug
>
>
> - Doug
>
>
> On Sat, Jun 27, 2015 at 4:54 PM, Aaron Ballman <aaron at aaronballman.com>
> wrote:
>
>> On Sat, Jun 27, 2015 at 10:45 AM, b17 c0de <b17c0de at gmail.com> wrote:
>> > __has_extension(nullability) returns true in non-GNU mode, on the other
>> > hand,__has_extension(assume_nonnull) returns false in non-GNU mode. Are
>> you
>> > saying this difference is by design. If so, why?
>>
>> I answered a bit too early in the morning. ;-) I forgot that
>> __has_extension inherits functionality from __has_feature. So
>> assume_nonnull should be true with either __has_feature or
>> __has_extension in GNU or ObjC mode, but is currently false in other
>> modes. You are correct that nullability is a bit different, and I'm
>> not certain why. __has_feature(nullability) will return true for GNU
>> and ObjC mode. __has_extension(nullability) will always return true.
>>
>> I am not certain whether this is by design or is a bug, but perhaps
>> Doug can explain. (I'm also a bit curious as to why GNU mode is
>> required.)
>>
>> ~Aaron
>>
>> >
>> > On Sat, Jun 27, 2015 at 4:35 PM, Aaron Ballman <aaron at aaronballman.com>
>> > wrote:
>> >>
>> >> On Sat, Jun 27, 2015 at 8:06 AM, b17 c0de <b17c0de at gmail.com> wrote:
>> >> > I figured out my issue. I was compiling with -std=c++14 and the
>> >> > nullability
>> >> > and assume_nonnull features are only enabled for ObjC and GNU mode.
>> Why
>> >> > are
>> >> > these only supported in GNU mode? I thought GNU mode was only for
>> >> > features
>> >> > that contradict the standard. How does this feature contradict the
>> >> > standard
>> >> > given that the names are double and single underscore prefixed? I
>> would
>> >> > rather not have to compile my code in GNU mode just to enable
>> >> > nullability. I
>> >> > can check with __has_extension() but at least Apple headers seem to
>> only
>> >> > use
>> >> > __has_feature so the checks there won't be enabled when not
>> compiling in
>> >> > GNU
>> >> > mode. If the consensus is that __has_feature(nullability) should
>> only be
>> >> > enabled for GNU mode, would it make sense to have an f-group flag
>> like
>> >> > -fnullability to enable it for __has_feature when not compiling in
>> GNU
>> >> > mode?
>> >> >
>> >> > Also I found a bug in clang. __has_extension(assume_nonnull) doesn't
>> >> > work
>> >> > properly. It is missing from the StringCase at the end of the
>> >> > HasExtension()
>> >> > function in lib/Lex/PPMacroExpansion.cpp. I think it should be there.
>> >>
>> >> __has_feature(assume_nonnull) is the way to test for that feature
>> >> (which is also GNU and Obj-C only).
>> >>
>> >> ~Aaron
>> >>
>> >> >
>> >> > On Sat, Jun 27, 2015 at 12:44 AM, b17 c0de <b17c0de at gmail.com>
>> wrote:
>> >> >>
>> >> >> Apple please implement  __has_feature(nullability) in clang for
>> Xcode 7
>> >> >> release. :-)
>> >> >>
>> >> >>
>> >> >> On Sat, Jun 27, 2015 at 12:31 AM, Aaron Ballman
>> >> >> <aaron at aaronballman.com>
>> >> >> wrote:
>> >> >>>
>> >> >>> On Fri, Jun 26, 2015 at 6:29 PM, b17 c0de <b17c0de at gmail.com>
>> wrote:
>> >> >>> >  It also appears that the current versions of Apple clang (even
>> the
>> >> >>> > newest
>> >> >>> > beta) don't even support __has_feature(nullability). I take it
>> this
>> >> >>> > has
>> >> >>> > been
>> >> >>> > fixed in trunk?
>> >> >>>
>> >> >>> Correct, trunk is likely also the only place that has _Nonnull and
>> >> >>> friends, too. If you're developing on OS X and don't need cross
>> >> >>> compiler support for your code base, I would stick with __nonnull
>> >> >>> there and you'll be fine. If you need cross compiler support,
>> you'll
>> >> >>> likely have to piece it together with macros.
>> >> >>>
>> >> >>> ~Aaron
>> >> >>>
>> >> >>> >
>> >> >>> > On Fri, Jun 26, 2015 at 11:53 PM, Aaron Ballman
>> >> >>> > <aaron at aaronballman.com>
>> >> >>> > wrote:
>> >> >>> >>
>> >> >>> >> On Fri, Jun 26, 2015 at 5:44 PM, b17 c0de <b17c0de at gmail.com>
>> >> >>> >> wrote:
>> >> >>> >> > OK. What would be the best way to detect if Apple clang
>> supports
>> >> >>> >> > _Nonnull or
>> >> >>> >> > only __nonnull though.
>> >> >>> >>
>> >> >>> >> I cannot speak for how Apple's Clang works in this regard, but
>> >> >>> >> perhaps
>> >> >>> >> Doug can.
>> >> >>> >>
>> >> >>> >> ~Aaron
>> >> >>> >>
>> >> >>> >> >
>> >> >>> >> > On Fri, Jun 26, 2015 at 11:40 PM, Aaron Ballman
>> >> >>> >> > <aaron at aaronballman.com>
>> >> >>> >> > wrote:
>> >> >>> >> >>
>> >> >>> >> >> On Fri, Jun 26, 2015 at 5:36 PM, b17 c0de <b17c0de at gmail.com
>> >
>> >> >>> >> >> wrote:
>> >> >>> >> >> > How can one detect if an Apple clang supports the new
>> >> >>> >> >> > nullability
>> >> >>> >> >> > attributes. I tried something like:
>> >> >>> >> >> >
>> >> >>> >> >> > #if __has_attribute(_Nonnull)
>> >> >>> >> >> > #elif __has_attribute(__nonnull)
>> >> >>> >> >> > #define _Nonnull __nonnull
>> >> >>> >> >> > #else
>> >> >>> >> >> > #define _Nonnull
>> >> >>> >> >> > #endif
>> >> >>> >> >> >
>> >> >>> >> >> > But this didn't work. Why doesn't _Nonnull/__nonnull work
>> with
>> >> >>> >> >> > __has_attribute?
>> >> >>> >> >>
>> >> >>> >> >> __has_attribute is used to test for GNU-style attribute
>> support
>> >> >>> >> >> only.
>> >> >>> >> >> To test for nullability, you should use:
>> >> >>> >> >> __has_feature(nullability)
>> >> >>> >> >>
>> >> >>> >> >> ~Aaron
>> >> >>> >> >>
>> >> >>> >> >> >
>> >> >>> >> >> > On Wed, Jun 24, 2015 at 10:39 PM, Douglas Gregor
>> >> >>> >> >> > <dgregor at apple.com>
>> >> >>> >> >> > wrote:
>> >> >>> >> >> >>
>> >> >>> >> >> >> Another addendum: due to the conflict with glibc’s
>> __nonnull,
>> >> >>> >> >> >> we’ll
>> >> >>> >> >> >> be
>> >> >>> >> >> >> renaming the __double_underscored keywords to
>> >> >>> >> >> >> _Big_underscored
>> >> >>> >> >> >> keywords,
>> >> >>> >> >> >> e.g.,
>> >> >>> >> >> >>
>> >> >>> >> >> >> __nonnull -> _Nonnull
>> >> >>> >> >> >> __nullable -> _Nullable
>> >> >>> >> >> >> __null_unspecified -> _Null_unspecified
>> >> >>> >> >> >>
>> >> >>> >> >> >> On Darwin, we’ll add predefines
>> >> >>> >> >> >>
>> >> >>> >> >> >> #define __nonnull _Nonnull
>> >> >>> >> >> >> #define __nullable _Nullable
>> >> >>> >> >> >> #define __null_unspecified _Null_unspecified
>> >> >>> >> >> >>
>> >> >>> >> >> >> to keep the existing headers working.
>> >> >>> >> >> >>
>> >> >>> >> >> >> - Doug
>> >> >>> >> >> >>
>> >> >>> >> >> >> On Mar 2, 2015, at 1:22 PM, Douglas Gregor
>> >> >>> >> >> >> <dgregor at apple.com>
>> >> >>> >> >> >> wrote:
>> >> >>> >> >> >>
>> >> >>> >> >> >> Hello all,
>> >> >>> >> >> >>
>> >> >>> >> >> >> Null pointers are a significant source of problems in
>> >> >>> >> >> >> applications.
>> >> >>> >> >> >> Whether it’s SIGSEGV taking down a process or a foolhardy
>> >> >>> >> >> >> attempt to
>> >> >>> >> >> >> recover
>> >> >>> >> >> >> from NullPointerException breaking invariants everywhere,
>> >> >>> >> >> >> it’s a
>> >> >>> >> >> >> problem
>> >> >>> >> >> >> that’s bad enough for Tony Hoare to call the invention of
>> the
>> >> >>> >> >> >> null
>> >> >>> >> >> >> reference
>> >> >>> >> >> >> his billion dollar mistake [1]. It’s not the ability to
>> >> >>> >> >> >> create a
>> >> >>> >> >> >> null
>> >> >>> >> >> >> pointer that is a problem—having a common sentinel value
>> >> >>> >> >> >> meaning
>> >> >>> >> >> >> “no
>> >> >>> >> >> >> value”
>> >> >>> >> >> >> is extremely useful—but that it’s very hard to determine
>> >> >>> >> >> >> whether,
>> >> >>> >> >> >> for a
>> >> >>> >> >> >> particular pointer, one is expected to be able to use
>> null. C
>> >> >>> >> >> >> doesn’t
>> >> >>> >> >> >> distinguish between “nullable” and “nonnull” pointers, so
>> we
>> >> >>> >> >> >> turn to
>> >> >>> >> >> >> documentation and experimentation. Consider strchr from
>> the C
>> >> >>> >> >> >> standard
>> >> >>> >> >> >> library:
>> >> >>> >> >> >>
>> >> >>> >> >> >> char *strchr(const char *s, int c);
>> >> >>> >> >> >>
>> >> >>> >> >> >> It is “obvious” to a programmer who knows the semantics of
>> >> >>> >> >> >> strchr
>> >> >>> >> >> >> that
>> >> >>> >> >> >> it’s important to check for a returned null, because null
>> is
>> >> >>> >> >> >> used as
>> >> >>> >> >> >> the
>> >> >>> >> >> >> sentinel for “not found”. Of course, your tools don’t know
>> >> >>> >> >> >> that,
>> >> >>> >> >> >> so
>> >> >>> >> >> >> they
>> >> >>> >> >> >> cannot help when you completely forget to check for the
>> null
>> >> >>> >> >> >> case.
>> >> >>> >> >> >> Bugs
>> >> >>> >> >> >> ensue.
>> >> >>> >> >> >>
>> >> >>> >> >> >> Can I pass a null string to strchr? The standard is
>> unclear
>> >> >>> >> >> >> [2],
>> >> >>> >> >> >> and
>> >> >>> >> >> >> my
>> >> >>> >> >> >> platform’s implementation happily accepts a null parameter
>> >> >>> >> >> >> and
>> >> >>> >> >> >> returns
>> >> >>> >> >> >> null,
>> >> >>> >> >> >> so obviously I shouldn’t worry about it… until I port my
>> >> >>> >> >> >> code,
>> >> >>> >> >> >> or
>> >> >>> >> >> >> the
>> >> >>> >> >> >> underlying implementation changes because my expectations
>> and
>> >> >>> >> >> >> the
>> >> >>> >> >> >> library
>> >> >>> >> >> >> implementor’s expectations differ. Given the age of
>> strchr, I
>> >> >>> >> >> >> suspect
>> >> >>> >> >> >> that
>> >> >>> >> >> >> every implementation out there has an explicit, defensive
>> >> >>> >> >> >> check
>> >> >>> >> >> >> for
>> >> >>> >> >> >> a
>> >> >>> >> >> >> null
>> >> >>> >> >> >> string, because it’s easier to add yet more defensive (and
>> >> >>> >> >> >> generally
>> >> >>> >> >> >> useless) null checks than it is to ask your clients to fix
>> >> >>> >> >> >> their
>> >> >>> >> >> >> code.
>> >> >>> >> >> >> Scale
>> >> >>> >> >> >> this up, and code bloat ensues, as well as wasted
>> programmer
>> >> >>> >> >> >> effort
>> >> >>> >> >> >> that
>> >> >>> >> >> >> obscures the places where checking for null really does
>> >> >>> >> >> >> matter.
>> >> >>> >> >> >>
>> >> >>> >> >> >> In a recent version of Xcode, Apple introduced an
>> extension
>> >> >>> >> >> >> to
>> >> >>> >> >> >> C/C++/Objective-C that expresses the nullability of
>> pointers
>> >> >>> >> >> >> in
>> >> >>> >> >> >> the
>> >> >>> >> >> >> type
>> >> >>> >> >> >> system via new nullability qualifiers . Nullability
>> >> >>> >> >> >> qualifiers
>> >> >>> >> >> >> express
>> >> >>> >> >> >> nullability as part of the declaration of strchr  [2]:
>> >> >>> >> >> >>
>> >> >>> >> >> >> __nullable char *strchr(__nonnull const char *s, int c);
>> >> >>> >> >> >>
>> >> >>> >> >> >> With this, programmers and tools alike can better reason
>> >> >>> >> >> >> about
>> >> >>> >> >> >> the
>> >> >>> >> >> >> use
>> >> >>> >> >> >> of
>> >> >>> >> >> >> strchr with null pointers.
>> >> >>> >> >> >>
>> >> >>> >> >> >> We’d like to contribute the implementation (and there is a
>> >> >>> >> >> >> patch
>> >> >>> >> >> >> attached
>> >> >>> >> >> >> at the end [3]), but since this is a nontrivial extension
>> to
>> >> >>> >> >> >> all
>> >> >>> >> >> >> of
>> >> >>> >> >> >> the
>> >> >>> >> >> >> C
>> >> >>> >> >> >> family of languages that Clang supports, we believe that
>> it
>> >> >>> >> >> >> needs to
>> >> >>> >> >> >> be
>> >> >>> >> >> >> discussed here first.
>> >> >>> >> >> >>
>> >> >>> >> >> >> Goals
>> >> >>> >> >> >> We have several specific goals that informed the design of
>> >> >>> >> >> >> this
>> >> >>> >> >> >> feature.
>> >> >>> >> >> >>
>> >> >>> >> >> >> Allow the intended nullability to be expressed on all
>> >> >>> >> >> >> pointers:
>> >> >>> >> >> >> Pointers
>> >> >>> >> >> >> are used throughout library interfaces, and the
>> nullability
>> >> >>> >> >> >> of
>> >> >>> >> >> >> those
>> >> >>> >> >> >> pointers is an important part of the API contract with
>> users.
>> >> >>> >> >> >> It’s
>> >> >>> >> >> >> too
>> >> >>> >> >> >> simplistic to only allow function parameters to have
>> >> >>> >> >> >> nullability,
>> >> >>> >> >> >> for
>> >> >>> >> >> >> example, because it’s also important information for data
>> >> >>> >> >> >> members,
>> >> >>> >> >> >> pointers-to-pointers (e.g., "a nonnull pointer to a
>> nullable
>> >> >>> >> >> >> pointer
>> >> >>> >> >> >> to
>> >> >>> >> >> >> an
>> >> >>> >> >> >> integer”), arrays of pointers, etc.
>> >> >>> >> >> >> Enable better tools support for detecting nullability
>> >> >>> >> >> >> problems:
>> >> >>> >> >> >> The
>> >> >>> >> >> >> nullability annotations should be useful for tools
>> >> >>> >> >> >> (especially
>> >> >>> >> >> >> the
>> >> >>> >> >> >> static
>> >> >>> >> >> >> analyzer) that can reason about the use of null, to give
>> >> >>> >> >> >> warnings
>> >> >>> >> >> >> about
>> >> >>> >> >> >> both
>> >> >>> >> >> >> missed null checks (the result of strchr could be null…)
>> as
>> >> >>> >> >> >> well
>> >> >>> >> >> >> as
>> >> >>> >> >> >> for
>> >> >>> >> >> >> unnecessarily-defensive code.
>> >> >>> >> >> >> Support workflows where all interfaces provide nullability
>> >> >>> >> >> >> annotations:
>> >> >>> >> >> >> In
>> >> >>> >> >> >> moving from a world where there are no nullability
>> >> >>> >> >> >> annotations
>> >> >>> >> >> >> to
>> >> >>> >> >> >> one
>> >> >>> >> >> >> where
>> >> >>> >> >> >> we hope to see many such annotations, we’ve found it
>> helpful
>> >> >>> >> >> >> to
>> >> >>> >> >> >> move
>> >> >>> >> >> >> header-by-header, auditing a complete header to give it
>> >> >>> >> >> >> nullability
>> >> >>> >> >> >> qualifiers. Once one has done that, additions to the
>> header
>> >> >>> >> >> >> need
>> >> >>> >> >> >> to
>> >> >>> >> >> >> be
>> >> >>> >> >> >> held
>> >> >>> >> >> >> to the same standard, so we need a design that allows us
>> to
>> >> >>> >> >> >> warn
>> >> >>> >> >> >> about
>> >> >>> >> >> >> pointers that don’t provide nullability annotations for
>> some
>> >> >>> >> >> >> declarations in
>> >> >>> >> >> >> a header that already has some nullability annotations.
>> >> >>> >> >> >>
>> >> >>> >> >> >> Zero effect on ABI or code generation: There are a huge
>> >> >>> >> >> >> number
>> >> >>> >> >> >> of
>> >> >>> >> >> >> interfaces that could benefit from the use of nullability
>> >> >>> >> >> >> qualifiers,
>> >> >>> >> >> >> but we
>> >> >>> >> >> >> won’t get widespread adoption if introducing the
>> nullability
>> >> >>> >> >> >> qualifiers
>> >> >>> >> >> >> means breaking existing code, either in the ABI (say,
>> because
>> >> >>> >> >> >> nullability
>> >> >>> >> >> >> qualifiers are mangled into the type) or at execution time
>> >> >>> >> >> >> (e.g.,
>> >> >>> >> >> >> because a
>> >> >>> >> >> >> non-null pointer ends up being null along some error path
>> and
>> >> >>> >> >> >> causes
>> >> >>> >> >> >> undefined behavior).
>> >> >>> >> >> >>
>> >> >>> >> >> >>
>> >> >>> >> >> >>
>> >> >>> >> >> >>
>> >> >>> >> >> >> Why not __attribute__((nonnull))?
>> >> >>> >> >> >> Clang already has an attribute to express nullability,
>> >> >>> >> >> >> “nonnull”,
>> >> >>> >> >> >> which
>> >> >>> >> >> >> we
>> >> >>> >> >> >> inherited from GCC [4]. The “nonnull” attribute can be
>> placed
>> >> >>> >> >> >> on
>> >> >>> >> >> >> functions
>> >> >>> >> >> >> to indicate which parameters cannot be null: one either
>> >> >>> >> >> >> specifies
>> >> >>> >> >> >> the
>> >> >>> >> >> >> indices of the arguments that cannot be null, e.g.,
>> >> >>> >> >> >>
>> >> >>> >> >> >> extern void *my_memcpy (void *dest, const void *src,
>> size_t
>> >> >>> >> >> >> len)
>> >> >>> >> >> >> __attribute__((nonnull (1, 2)));
>> >> >>> >> >> >>
>> >> >>> >> >> >> or omits the list of indices to state that all pointer
>> >> >>> >> >> >> arguments
>> >> >>> >> >> >> cannot
>> >> >>> >> >> >> be
>> >> >>> >> >> >> null, e.g.,
>> >> >>> >> >> >>
>> >> >>> >> >> >> extern void *my_memcpy (void *dest, const void *src,
>> size_t
>> >> >>> >> >> >> len)
>> >> >>> >> >> >> __attribute__((nonnull));
>> >> >>> >> >> >>
>> >> >>> >> >> >> More recently, “nonnull”  has grown the ability to be
>> applied
>> >> >>> >> >> >> to
>> >> >>> >> >> >> parameters, and one can use the companion attribute
>> >> >>> >> >> >> returns_nonnull
>> >> >>> >> >> >> to
>> >> >>> >> >> >> state
>> >> >>> >> >> >> that a function returns a non-null pointer:
>> >> >>> >> >> >>
>> >> >>> >> >> >> extern void *my_memcpy (__attribute__((nonnull)) void
>> *dest,
>> >> >>> >> >> >> __attribute__((nonnull)) const void *src, size_t len)
>> >> >>> >> >> >> __attribute__((returns_nonnull));
>> >> >>> >> >> >>
>> >> >>> >> >> >> There are a number of problems here. First, there are
>> >> >>> >> >> >> different
>> >> >>> >> >> >> attributes
>> >> >>> >> >> >> to express the same idea at different places in the
>> grammar,
>> >> >>> >> >> >> and
>> >> >>> >> >> >> the
>> >> >>> >> >> >> use of
>> >> >>> >> >> >> the “nonnull” attribute on the function actually has an
>> >> >>> >> >> >> effect
>> >> >>> >> >> >> on
>> >> >>> >> >> >> the
>> >> >>> >> >> >> function parameters can get very, very confusing. Quick,
>> >> >>> >> >> >> which
>> >> >>> >> >> >> pointers
>> >> >>> >> >> >> are
>> >> >>> >> >> >> nullable vs. non-null in this example?
>> >> >>> >> >> >>
>> >> >>> >> >> >> __attribute__((nonnull)) void *my_realloc (void *ptr,
>> size_t
>> >> >>> >> >> >> size);
>> >> >>> >> >> >>
>> >> >>> >> >> >> According to that declaration, ptr is nonnull and the
>> >> >>> >> >> >> function
>> >> >>> >> >> >> returns
>> >> >>> >> >> >> a
>> >> >>> >> >> >> nullable pointer… but that’s the opposite of how it reads
>> >> >>> >> >> >> (and
>> >> >>> >> >> >> behaves,
>> >> >>> >> >> >> if
>> >> >>> >> >> >> this is anything like a realloc that cannot fail).
>> Moreover,
>> >> >>> >> >> >> because
>> >> >>> >> >> >> these
>> >> >>> >> >> >> two attributes are declaration attributes, not type
>> >> >>> >> >> >> attributes,
>> >> >>> >> >> >> you
>> >> >>> >> >> >> cannot
>> >> >>> >> >> >> express that nullability of the inner pointer in a
>> >> >>> >> >> >> multi-level
>> >> >>> >> >> >> pointer
>> >> >>> >> >> >> or an
>> >> >>> >> >> >> array of pointers, which makes these attributes verbose,
>> >> >>> >> >> >> confusing,
>> >> >>> >> >> >> and
>> >> >>> >> >> >> not
>> >> >>> >> >> >> sufficiently generally. These attributes fail the first of
>> >> >>> >> >> >> our
>> >> >>> >> >> >> goals.
>> >> >>> >> >> >>
>> >> >>> >> >> >> These attributes aren’t as useful as they could be for
>> tools
>> >> >>> >> >> >> support
>> >> >>> >> >> >> (the
>> >> >>> >> >> >> second and third goals), because they only express the
>> >> >>> >> >> >> nonnull
>> >> >>> >> >> >> case,
>> >> >>> >> >> >> leaving
>> >> >>> >> >> >> no way to distinguish between the unannotated case (nobody
>> >> >>> >> >> >> has
>> >> >>> >> >> >> documented
>> >> >>> >> >> >> the nullability of some parameter) and the nullable case
>> (we
>> >> >>> >> >> >> know
>> >> >>> >> >> >> the
>> >> >>> >> >> >> pointer can be null). From a tooling perspective, this is
>> a
>> >> >>> >> >> >> killer:
>> >> >>> >> >> >> the
>> >> >>> >> >> >> static analyzer absolutely cannot warn that one has
>> forgotten
>> >> >>> >> >> >> to
>> >> >>> >> >> >> check
>> >> >>> >> >> >> for
>> >> >>> >> >> >> null for every unannotated pointer, because the
>> >> >>> >> >> >> false-positive
>> >> >>> >> >> >> rate
>> >> >>> >> >> >> would be
>> >> >>> >> >> >> astronomical.
>> >> >>> >> >> >>
>> >> >>> >> >> >> Finally, we’ve recently started considering violations of
>> the
>> >> >>> >> >> >> __attribute__((nonnull)) contract to be undefined
>> behavior,
>> >> >>> >> >> >> which
>> >> >>> >> >> >> fails
>> >> >>> >> >> >> the
>> >> >>> >> >> >> last of our goals. This is something we could debate
>> further
>> >> >>> >> >> >> if
>> >> >>> >> >> >> it
>> >> >>> >> >> >> were
>> >> >>> >> >> >> the
>> >> >>> >> >> >> only problem, but these declaration attributes fall all of
>> >> >>> >> >> >> our
>> >> >>> >> >> >> criteria, so
>> >> >>> >> >> >> it’s not worth discussing.
>> >> >>> >> >> >>
>> >> >>> >> >> >> Nullability Qualifiers
>> >> >>> >> >> >> We propose the addition of a new set of type qualifiers,
>> >> >>> >> >> >> spelled
>> >> >>> >> >> >> __nullable, __nonnull, and __null_unspecified, to Clang.
>> >> >>> >> >> >> These
>> >> >>> >> >> >> are
>> >> >>> >> >> >> collectively known as nullability qualifiers and may be
>> >> >>> >> >> >> written
>> >> >>> >> >> >> anywhere any
>> >> >>> >> >> >> other type qualifier may be written (such as const) on any
>> >> >>> >> >> >> type
>> >> >>> >> >> >> subject
>> >> >>> >> >> >> to
>> >> >>> >> >> >> the following restrictions:
>> >> >>> >> >> >>
>> >> >>> >> >> >> Two nullability qualifiers shall not appear in the same
>> set
>> >> >>> >> >> >> of
>> >> >>> >> >> >> qualifiers.
>> >> >>> >> >> >> A nullability qualifier shall qualify any pointer type,
>> >> >>> >> >> >> including
>> >> >>> >> >> >> pointers
>> >> >>> >> >> >> to objects, pointers to functions, C++ pointers to
>> members,
>> >> >>> >> >> >> block
>> >> >>> >> >> >> pointers,
>> >> >>> >> >> >> and Objective-C object pointers.
>> >> >>> >> >> >> A nullability qualifier in the declaration-specifiers
>> applies
>> >> >>> >> >> >> to
>> >> >>> >> >> >> the
>> >> >>> >> >> >> innermost pointer type of each declarator (e.g., __nonnull
>> >> >>> >> >> >> int *
>> >> >>> >> >> >> is
>> >> >>> >> >> >> equivalent to int * __nonnull).
>> >> >>> >> >> >> A nullability qualifier applied to a typedef of a
>> >> >>> >> >> >> nullability-qualified
>> >> >>> >> >> >> pointer type shall specify the same nullability as the
>> >> >>> >> >> >> underlying
>> >> >>> >> >> >> type
>> >> >>> >> >> >> of
>> >> >>> >> >> >> the typedef.
>> >> >>> >> >> >>
>> >> >>> >> >> >>
>> >> >>> >> >> >> The meanings of the three nullability qualifiers are as
>> >> >>> >> >> >> follows:
>> >> >>> >> >> >>
>> >> >>> >> >> >> __nullable: the pointer may store a null value at runtime
>> (as
>> >> >>> >> >> >> part
>> >> >>> >> >> >> of
>> >> >>> >> >> >> the
>> >> >>> >> >> >> API contract)
>> >> >>> >> >> >> __nonnull: the pointer should not store a null value at
>> >> >>> >> >> >> runtime
>> >> >>> >> >> >> (as
>> >> >>> >> >> >> part
>> >> >>> >> >> >> of the API contract). it is possible that the value can be
>> >> >>> >> >> >> null,
>> >> >>> >> >> >> e.g.,
>> >> >>> >> >> >> in
>> >> >>> >> >> >> erroneous historic uses of an API, and it is up to the
>> >> >>> >> >> >> library
>> >> >>> >> >> >> implementor
>> >> >>> >> >> >> to decide to what degree she will accommodate such
>> clients.
>> >> >>> >> >> >> __null_unspecified: it is unclear whether the pointer can
>> be
>> >> >>> >> >> >> null or
>> >> >>> >> >> >> not.
>> >> >>> >> >> >> Use of this type qualifier is extremely rare in practice,
>> but
>> >> >>> >> >> >> it
>> >> >>> >> >> >> fills
>> >> >>> >> >> >> a
>> >> >>> >> >> >> small but important niche when auditing a particular
>> header
>> >> >>> >> >> >> to
>> >> >>> >> >> >> add
>> >> >>> >> >> >> nullability qualifiers: sometimes the nullability contract
>> >> >>> >> >> >> for a
>> >> >>> >> >> >> few
>> >> >>> >> >> >> APIs in
>> >> >>> >> >> >> the header is unclear even when looking at the
>> implementation
>> >> >>> >> >> >> for
>> >> >>> >> >> >> historical
>> >> >>> >> >> >> reasons, and establishing the contract requires more
>> >> >>> >> >> >> extensive
>> >> >>> >> >> >> study.
>> >> >>> >> >> >> In
>> >> >>> >> >> >> such cases, it’s often best to mark that pointer as
>> >> >>> >> >> >> __null_unspecified
>> >> >>> >> >> >> (which will help silence the warning about unannotated
>> >> >>> >> >> >> pointers
>> >> >>> >> >> >> in a
>> >> >>> >> >> >> header)
>> >> >>> >> >> >> and move on, coming back to __null_unspecified pointers
>> when
>> >> >>> >> >> >> the
>> >> >>> >> >> >> appropriate
>> >> >>> >> >> >> graybeard has been summoned out of retirement [5].
>> >> >>> >> >> >>
>> >> >>> >> >> >> Assumes-nonnull Regions
>> >> >>> >> >> >> We’ve found that it's fairly common for the majority of
>> >> >>> >> >> >> pointers
>> >> >>> >> >> >> within
>> >> >>> >> >> >> a
>> >> >>> >> >> >> particular header to be __nonnull. Therefore, we’ve
>> >> >>> >> >> >> introduced
>> >> >>> >> >> >> assumes-nonnull regions that assume that certain
>> unannotated
>> >> >>> >> >> >> pointers
>> >> >>> >> >> >> implicitly get the __nonnull nullability qualifiers.
>> >> >>> >> >> >> Assumes-nonnull
>> >> >>> >> >> >> regions
>> >> >>> >> >> >> are marked by pragmas:
>> >> >>> >> >> >>
>> >> >>> >> >> >> #pragma clang assume_nonnull begin
>> >> >>> >> >> >>         __nullable char *strchr(const char *s, int c); //
>> s
>> >> >>> >> >> >> is
>> >> >>> >> >> >> inferred
>> >> >>> >> >> >> to
>> >> >>> >> >> >> be __nonnull
>> >> >>> >> >> >> void *my_realloc (__nullable void *ptr, size_t size); //
>> >> >>> >> >> >> my_realloc
>> >> >>> >> >> >> is
>> >> >>> >> >> >> inferred to return __nonnull
>> >> >>> >> >> >> #pragma clang assume_nonnull end
>> >> >>> >> >> >>
>> >> >>> >> >> >> We infer __nonnull within an assumes_nonnull region when:
>> >> >>> >> >> >>
>> >> >>> >> >> >> The pointer is a non-typedef declaration, such as a
>> function
>> >> >>> >> >> >> parameter,
>> >> >>> >> >> >> variable, or data member, or the result type of a
>> function.
>> >> >>> >> >> >> It’s
>> >> >>> >> >> >> very
>> >> >>> >> >> >> rare
>> >> >>> >> >> >> for one to warn typedefs to specify nullability
>> information;
>> >> >>> >> >> >> rather,
>> >> >>> >> >> >> it’s
>> >> >>> >> >> >> usually the user of the typedef that needs to specify
>> >> >>> >> >> >> nullability.
>> >> >>> >> >> >> The pointer is a single-level pointer, e.g., int* but not
>> >> >>> >> >> >> int**,
>> >> >>> >> >> >> because
>> >> >>> >> >> >> we’ve found that programmers can get confused about the
>> >> >>> >> >> >> nullability
>> >> >>> >> >> >> of
>> >> >>> >> >> >> multi-level pointers (is it a __nullable pointer to
>> __nonnull
>> >> >>> >> >> >> pointers,
>> >> >>> >> >> >> or
>> >> >>> >> >> >> the other way around?) and inferring nullability for any
>> of
>> >> >>> >> >> >> the
>> >> >>> >> >> >> pointers in
>> >> >>> >> >> >> a multi-level pointer compounds the situation.
>> >> >>> >> >> >>
>> >> >>> >> >> >>
>> >> >>> >> >> >> Note that no #include may occur within an assumes_nonnull
>> >> >>> >> >> >> region,
>> >> >>> >> >> >> and
>> >> >>> >> >> >> assumes_nonnull regions cannot cross header boundaries.
>> >> >>> >> >> >>
>> >> >>> >> >> >> Type System Impact
>> >> >>> >> >> >> Nullability qualifiers are mapped to type attributes
>> within
>> >> >>> >> >> >> the
>> >> >>> >> >> >> Clang
>> >> >>> >> >> >> type
>> >> >>> >> >> >> system, but a nullability-qualified pointer type is not
>> >> >>> >> >> >> semantically
>> >> >>> >> >> >> distinct from its unqualified pointer type. Therefore, one
>> >> >>> >> >> >> may
>> >> >>> >> >> >> freely
>> >> >>> >> >> >> convert between nullability-qualified and
>> >> >>> >> >> >> non-nullability-qualified
>> >> >>> >> >> >> pointers, or between nullability-qualified pointers with
>> >> >>> >> >> >> different
>> >> >>> >> >> >> nullability qualifiers. One cannot overload on nullability
>> >> >>> >> >> >> qualifiers,
>> >> >>> >> >> >> write
>> >> >>> >> >> >> C++ class template partial specializations that identify
>> >> >>> >> >> >> nullability
>> >> >>> >> >> >> qualifiers, or inspect nullability via type traits in any
>> >> >>> >> >> >> way.
>> >> >>> >> >> >>
>> >> >>> >> >> >> Said more strongly, removing nullability qualifiers from a
>> >> >>> >> >> >> well-formed
>> >> >>> >> >> >> program will not change its behavior in any way, nor will
>> the
>> >> >>> >> >> >> semantics
>> >> >>> >> >> >> of a
>> >> >>> >> >> >> program change when any set of (well-formed) nullability
>> >> >>> >> >> >> qualifiers
>> >> >>> >> >> >> are
>> >> >>> >> >> >> added to it. Operationally, this means that nullability
>> >> >>> >> >> >> qualifiers
>> >> >>> >> >> >> are
>> >> >>> >> >> >> not
>> >> >>> >> >> >> part of the canonical type in Clang’s type system, and
>> that
>> >> >>> >> >> >> any
>> >> >>> >> >> >> warnings we
>> >> >>> >> >> >> produce based on nullability information will necessarily
>> be
>> >> >>> >> >> >> dependent
>> >> >>> >> >> >> on
>> >> >>> >> >> >> Clang’s ability to retain type sugar during semantic
>> >> >>> >> >> >> analysis.
>> >> >>> >> >> >>
>> >> >>> >> >> >> While it’s somewhat exceptional for us to introduce new
>> type
>> >> >>> >> >> >> qualifiers
>> >> >>> >> >> >> that don’t produce semantically distinct types, we feel
>> that
>> >> >>> >> >> >> this is
>> >> >>> >> >> >> the
>> >> >>> >> >> >> only plausible design and implementation strategy for this
>> >> >>> >> >> >> feature:
>> >> >>> >> >> >> pushing
>> >> >>> >> >> >> nullability qualifiers into the type system semantically
>> >> >>> >> >> >> would
>> >> >>> >> >> >> cause
>> >> >>> >> >> >> significant changes to the language (e.g., overloading,
>> >> >>> >> >> >> partial
>> >> >>> >> >> >> specialization) and break ABI (due to name mangling) that
>> >> >>> >> >> >> would
>> >> >>> >> >> >> drastically
>> >> >>> >> >> >> reduce the number of potential users, and we feel that
>> >> >>> >> >> >> Clang’s
>> >> >>> >> >> >> support
>> >> >>> >> >> >> for
>> >> >>> >> >> >> maintaining type sugar throughout semantic analysis is
>> >> >>> >> >> >> generally
>> >> >>> >> >> >> good
>> >> >>> >> >> >> enough
>> >> >>> >> >> >> [6] to get the benefits of nullability annotations in our
>> >> >>> >> >> >> tools.
>> >> >>> >> >> >>
>> >> >>> >> >> >> Looking forward to our discussion.
>> >> >>> >> >> >>
>> >> >>> >> >> >> - Doug (with Jordan Rose and Anna Zaks)
>> >> >>> >> >> >>
>> >> >>> >> >> >> [1]
>> >> >>> >> >> >>
>> >> >>> >> >> >>
>> >> >>> >> >> >>
>> http://en.wikipedia.org/wiki/Tony_Hoare#Apologies_and_retractions
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__en.wikipedia.org_wiki_Tony-5FHoare-23Apologies-5Fand-5Fretractions&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=CnzuN65ENJ1H9py9XLiRvC_UQz6u3oG6GUNn7_wosSM&m=bljRv73Hdesdao5bJgcUiiE_nylTGrbpSZs4E6AE8Fo&s=_PIx9QaKZlsYZF-JpRLnkzkjNK6U-ip_DqRWv3Xrhcw&e=>
>> >> >>> >> >> >> [2] The standard description of strchr seems to imply that
>> >> >>> >> >> >> the
>> >> >>> >> >> >> parameter
>> >> >>> >> >> >> cannot be null
>> >> >>> >> >> >> [3] The patch is complete, but should be reviewed on
>> >> >>> >> >> >> cfe-commits
>> >> >>> >> >> >> rather
>> >> >>> >> >> >> than here. There are also several logic parts to this
>> >> >>> >> >> >> monolithic
>> >> >>> >> >> >> patch:
>> >> >>> >> >> >> (a) __nonnull/__nullable/__null_unspecified type
>> specifiers
>> >> >>> >> >> >> (b) nonnull/nullable/null_unspecified syntactic sugar for
>> >> >>> >> >> >> Objective-C
>> >> >>> >> >> >> (c) Warning about inconsistent application of nullability
>> >> >>> >> >> >> specifiers
>> >> >>> >> >> >> within a given header
>> >> >>> >> >> >> (d) assume_nonnnull begin/end pragmas
>> >> >>> >> >> >> (e) Objective-C null_resettable property attribute
>> >> >>> >> >> >> [4]
>> >> >>> >> >> >>
>> https://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__gcc.gnu.org_onlinedocs_gcc_Function-2DAttributes.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=CnzuN65ENJ1H9py9XLiRvC_UQz6u3oG6GUNn7_wosSM&m=bljRv73Hdesdao5bJgcUiiE_nylTGrbpSZs4E6AE8Fo&s=7XK1ONRGcffCS5T_y1QAAPCMo3IPe50NIVPvlF08JmM&e=>
>> >> >>> >> >> >> (search
>> >> >>> >> >> >> for “nonnull”)
>> >> >>> >> >> >> [5] No graybeards were harmed in the making of this
>> feature.
>> >> >>> >> >> >> [6] Template instantiation is the notable exception here,
>> >> >>> >> >> >> because it
>> >> >>> >> >> >> always canonicalizes types.
>> >> >>> >> >> >>
>> >> >>> >> >> >> <nullability.patch>
>> >> >>> >> >> >> _______________________________________________
>> >> >>> >> >> >> cfe-dev mailing list
>> >> >>> >> >> >> cfe-dev at cs.uiuc.edu
>> >> >>> >> >> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>> >> >>> >> >> >>
>> >> >>> >> >> >>
>> >> >>> >> >> >>
>> >> >>> >> >> >> _______________________________________________
>> >> >>> >> >> >> cfe-dev mailing list
>> >> >>> >> >> >> cfe-dev at cs.uiuc.edu
>> >> >>> >> >> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>> >> >>> >> >> >>
>> >> >>> >> >> >
>> >> >>> >> >> >
>> >> >>> >> >> > _______________________________________________
>> >> >>> >> >> > cfe-dev mailing list
>> >> >>> >> >> > cfe-dev at cs.uiuc.edu
>> >> >>> >> >> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>> >> >>> >> >> >
>> >> >>> >> >
>> >> >>> >> >
>> >> >>> >
>> >> >>> >
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > cfe-dev mailing list
>> >> > cfe-dev at cs.uiuc.edu
>> >> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>> >> >
>> >
>> >
>>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150629/603294fa/attachment.html>