[cfe-dev] RFC: Nullability qualifiers

[b17 c0de]

Filed bug 23987.

On Mon, Jun 29, 2015 at 10:05 PM, Douglas Gregor <dgregor at apple.com> wrote:

>
> On Jun 29, 2015, at 1:00 PM, b17 c0de <b17c0de at gmail.com> wrote:
>
> Hi Doug,
> Thanks for your changes. I think the choice you made makes the most sense.
>
> Can you look at the nullability-completeness bug with templated types? Or
> should I make a bug report?
>
>
> Please file a bug report. It helps us track fixes.
>
> - Doug
>
>
> On Mon, Jun 29, 2015 at 8:12 PM, Douglas Gregor <dgregor at apple.com> wrote:
>
>>
>> On Jun 29, 2015, at 10:57 AM, Douglas Gregor <dgregor at apple.com> wrote:
>>
>>
>> On Jun 27, 2015, at 8:03 AM, b17 c0de <b17c0de at gmail.com> wrote:
>>
>> Just to clarify...
>>
>> __has_extension(nullability) always returning true is exactly what I
>> would expect from reading the clang documentation. I would also expect that
>> __has_extension(assume_nonnull) always returns true (which it currently
>> doesn't).
>>
>>
>> __has_extension(assume_nonnull) should always be true; fixed in r240969.
>>
>> I am hoping someone can explain why GNU mode is required for
>> __has_feature(nullability) and __has_feature(assume_nonnull) to return
>> true.
>>
>>
>> GNU mode was meant to capture “the default behavior”, and gets disabled
>> when one specifies stricter conformance to the language standard. Rather,
>> nullability is an extension that is always available, and only a
>> “standardized” language feature in Objective-C (where Clang is essentially
>> the standard). In retrospect, we shouldn’t straddle the fence this way with
>> GNUMode: either it’s an always-on feature (__has_feature and
>> __has_extension are always true for both) or it’s only a language feature
>> in Objective-C (__has_feature is true only in Objective-C mode,
>> __has_extension is always true). I tend to prefer the former.
>>
>>
>> r240976 makes these always-true.
>>
>> - Doug
>>
>>
>> - Doug
>>
>>
>> On Sat, Jun 27, 2015 at 4:54 PM, Aaron Ballman <aaron at aaronballman.com>
>> wrote:
>>
>>> On Sat, Jun 27, 2015 at 10:45 AM, b17 c0de <b17c0de at gmail.com> wrote:
>>> > __has_extension(nullability) returns true in non-GNU mode, on the other
>>> > hand,__has_extension(assume_nonnull) returns false in non-GNU mode.
>>> Are you
>>> > saying this difference is by design. If so, why?
>>>
>>> I answered a bit too early in the morning. ;-) I forgot that
>>> __has_extension inherits functionality from __has_feature. So
>>> assume_nonnull should be true with either __has_feature or
>>> __has_extension in GNU or ObjC mode, but is currently false in other
>>> modes. You are correct that nullability is a bit different, and I'm
>>> not certain why. __has_feature(nullability) will return true for GNU
>>> and ObjC mode. __has_extension(nullability) will always return true.
>>>
>>> I am not certain whether this is by design or is a bug, but perhaps
>>> Doug can explain. (I'm also a bit curious as to why GNU mode is
>>> required.)
>>>
>>> ~Aaron
>>>
>>> >
>>> > On Sat, Jun 27, 2015 at 4:35 PM, Aaron Ballman <aaron at aaronballman.com
>>> >
>>> > wrote:
>>> >>
>>> >> On Sat, Jun 27, 2015 at 8:06 AM, b17 c0de <b17c0de at gmail.com> wrote:
>>> >> > I figured out my issue. I was compiling with -std=c++14 and the
>>> >> > nullability
>>> >> > and assume_nonnull features are only enabled for ObjC and GNU mode.
>>> Why
>>> >> > are
>>> >> > these only supported in GNU mode? I thought GNU mode was only for
>>> >> > features
>>> >> > that contradict the standard. How does this feature contradict the
>>> >> > standard
>>> >> > given that the names are double and single underscore prefixed? I
>>> would
>>> >> > rather not have to compile my code in GNU mode just to enable
>>> >> > nullability. I
>>> >> > can check with __has_extension() but at least Apple headers seem to
>>> only
>>> >> > use
>>> >> > __has_feature so the checks there won't be enabled when not
>>> compiling in
>>> >> > GNU
>>> >> > mode. If the consensus is that __has_feature(nullability) should
>>> only be
>>> >> > enabled for GNU mode, would it make sense to have an f-group flag
>>> like
>>> >> > -fnullability to enable it for __has_feature when not compiling in
>>> GNU
>>> >> > mode?
>>> >> >
>>> >> > Also I found a bug in clang. __has_extension(assume_nonnull) doesn't
>>> >> > work
>>> >> > properly. It is missing from the StringCase at the end of the
>>> >> > HasExtension()
>>> >> > function in lib/Lex/PPMacroExpansion.cpp. I think it should be
>>> there.
>>> >>
>>> >> __has_feature(assume_nonnull) is the way to test for that feature
>>> >> (which is also GNU and Obj-C only).
>>> >>
>>> >> ~Aaron
>>> >>
>>> >> >
>>> >> > On Sat, Jun 27, 2015 at 12:44 AM, b17 c0de <b17c0de at gmail.com>
>>> wrote:
>>> >> >>
>>> >> >> Apple please implement  __has_feature(nullability) in clang for
>>> Xcode 7
>>> >> >> release. :-)
>>> >> >>
>>> >> >>
>>> >> >> On Sat, Jun 27, 2015 at 12:31 AM, Aaron Ballman
>>> >> >> <aaron at aaronballman.com>
>>> >> >> wrote:
>>> >> >>>
>>> >> >>> On Fri, Jun 26, 2015 at 6:29 PM, b17 c0de <b17c0de at gmail.com>
>>> wrote:
>>> >> >>> >  It also appears that the current versions of Apple clang (even
>>> the
>>> >> >>> > newest
>>> >> >>> > beta) don't even support __has_feature(nullability). I take it
>>> this
>>> >> >>> > has
>>> >> >>> > been
>>> >> >>> > fixed in trunk?
>>> >> >>>
>>> >> >>> Correct, trunk is likely also the only place that has _Nonnull and
>>> >> >>> friends, too. If you're developing on OS X and don't need cross
>>> >> >>> compiler support for your code base, I would stick with __nonnull
>>> >> >>> there and you'll be fine. If you need cross compiler support,
>>> you'll
>>> >> >>> likely have to piece it together with macros.
>>> >> >>>
>>> >> >>> ~Aaron
>>> >> >>>
>>> >> >>> >
>>> >> >>> > On Fri, Jun 26, 2015 at 11:53 PM, Aaron Ballman
>>> >> >>> > <aaron at aaronballman.com>
>>> >> >>> > wrote:
>>> >> >>> >>
>>> >> >>> >> On Fri, Jun 26, 2015 at 5:44 PM, b17 c0de <b17c0de at gmail.com>
>>> >> >>> >> wrote:
>>> >> >>> >> > OK. What would be the best way to detect if Apple clang
>>> supports
>>> >> >>> >> > _Nonnull or
>>> >> >>> >> > only __nonnull though.
>>> >> >>> >>
>>> >> >>> >> I cannot speak for how Apple's Clang works in this regard, but
>>> >> >>> >> perhaps
>>> >> >>> >> Doug can.
>>> >> >>> >>
>>> >> >>> >> ~Aaron
>>> >> >>> >>
>>> >> >>> >> >
>>> >> >>> >> > On Fri, Jun 26, 2015 at 11:40 PM, Aaron Ballman
>>> >> >>> >> > <aaron at aaronballman.com>
>>> >> >>> >> > wrote:
>>> >> >>> >> >>
>>> >> >>> >> >> On Fri, Jun 26, 2015 at 5:36 PM, b17 c0de <
>>> b17c0de at gmail.com>
>>> >> >>> >> >> wrote:
>>> >> >>> >> >> > How can one detect if an Apple clang supports the new
>>> >> >>> >> >> > nullability
>>> >> >>> >> >> > attributes. I tried something like:
>>> >> >>> >> >> >
>>> >> >>> >> >> > #if __has_attribute(_Nonnull)
>>> >> >>> >> >> > #elif __has_attribute(__nonnull)
>>> >> >>> >> >> > #define _Nonnull __nonnull
>>> >> >>> >> >> > #else
>>> >> >>> >> >> > #define _Nonnull
>>> >> >>> >> >> > #endif
>>> >> >>> >> >> >
>>> >> >>> >> >> > But this didn't work. Why doesn't _Nonnull/__nonnull work
>>> with
>>> >> >>> >> >> > __has_attribute?
>>> >> >>> >> >>
>>> >> >>> >> >> __has_attribute is used to test for GNU-style attribute
>>> support
>>> >> >>> >> >> only.
>>> >> >>> >> >> To test for nullability, you should use:
>>> >> >>> >> >> __has_feature(nullability)
>>> >> >>> >> >>
>>> >> >>> >> >> ~Aaron
>>> >> >>> >> >>
>>> >> >>> >> >> >
>>> >> >>> >> >> > On Wed, Jun 24, 2015 at 10:39 PM, Douglas Gregor
>>> >> >>> >> >> > <dgregor at apple.com>
>>> >> >>> >> >> > wrote:
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Another addendum: due to the conflict with glibc’s
>>> __nonnull,
>>> >> >>> >> >> >> we’ll
>>> >> >>> >> >> >> be
>>> >> >>> >> >> >> renaming the __double_underscored keywords to
>>> >> >>> >> >> >> _Big_underscored
>>> >> >>> >> >> >> keywords,
>>> >> >>> >> >> >> e.g.,
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> __nonnull -> _Nonnull
>>> >> >>> >> >> >> __nullable -> _Nullable
>>> >> >>> >> >> >> __null_unspecified -> _Null_unspecified
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> On Darwin, we’ll add predefines
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> #define __nonnull _Nonnull
>>> >> >>> >> >> >> #define __nullable _Nullable
>>> >> >>> >> >> >> #define __null_unspecified _Null_unspecified
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> to keep the existing headers working.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> - Doug
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> On Mar 2, 2015, at 1:22 PM, Douglas Gregor
>>> >> >>> >> >> >> <dgregor at apple.com>
>>> >> >>> >> >> >> wrote:
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Hello all,
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Null pointers are a significant source of problems in
>>> >> >>> >> >> >> applications.
>>> >> >>> >> >> >> Whether it’s SIGSEGV taking down a process or a foolhardy
>>> >> >>> >> >> >> attempt to
>>> >> >>> >> >> >> recover
>>> >> >>> >> >> >> from NullPointerException breaking invariants everywhere,
>>> >> >>> >> >> >> it’s a
>>> >> >>> >> >> >> problem
>>> >> >>> >> >> >> that’s bad enough for Tony Hoare to call the invention
>>> of the
>>> >> >>> >> >> >> null
>>> >> >>> >> >> >> reference
>>> >> >>> >> >> >> his billion dollar mistake [1]. It’s not the ability to
>>> >> >>> >> >> >> create a
>>> >> >>> >> >> >> null
>>> >> >>> >> >> >> pointer that is a problem—having a common sentinel value
>>> >> >>> >> >> >> meaning
>>> >> >>> >> >> >> “no
>>> >> >>> >> >> >> value”
>>> >> >>> >> >> >> is extremely useful—but that it’s very hard to determine
>>> >> >>> >> >> >> whether,
>>> >> >>> >> >> >> for a
>>> >> >>> >> >> >> particular pointer, one is expected to be able to use
>>> null. C
>>> >> >>> >> >> >> doesn’t
>>> >> >>> >> >> >> distinguish between “nullable” and “nonnull” pointers,
>>> so we
>>> >> >>> >> >> >> turn to
>>> >> >>> >> >> >> documentation and experimentation. Consider strchr from
>>> the C
>>> >> >>> >> >> >> standard
>>> >> >>> >> >> >> library:
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> char *strchr(const char *s, int c);
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> It is “obvious” to a programmer who knows the semantics
>>> of
>>> >> >>> >> >> >> strchr
>>> >> >>> >> >> >> that
>>> >> >>> >> >> >> it’s important to check for a returned null, because
>>> null is
>>> >> >>> >> >> >> used as
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> sentinel for “not found”. Of course, your tools don’t
>>> know
>>> >> >>> >> >> >> that,
>>> >> >>> >> >> >> so
>>> >> >>> >> >> >> they
>>> >> >>> >> >> >> cannot help when you completely forget to check for the
>>> null
>>> >> >>> >> >> >> case.
>>> >> >>> >> >> >> Bugs
>>> >> >>> >> >> >> ensue.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Can I pass a null string to strchr? The standard is
>>> unclear
>>> >> >>> >> >> >> [2],
>>> >> >>> >> >> >> and
>>> >> >>> >> >> >> my
>>> >> >>> >> >> >> platform’s implementation happily accepts a null
>>> parameter
>>> >> >>> >> >> >> and
>>> >> >>> >> >> >> returns
>>> >> >>> >> >> >> null,
>>> >> >>> >> >> >> so obviously I shouldn’t worry about it… until I port my
>>> >> >>> >> >> >> code,
>>> >> >>> >> >> >> or
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> underlying implementation changes because my
>>> expectations and
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> library
>>> >> >>> >> >> >> implementor’s expectations differ. Given the age of
>>> strchr, I
>>> >> >>> >> >> >> suspect
>>> >> >>> >> >> >> that
>>> >> >>> >> >> >> every implementation out there has an explicit, defensive
>>> >> >>> >> >> >> check
>>> >> >>> >> >> >> for
>>> >> >>> >> >> >> a
>>> >> >>> >> >> >> null
>>> >> >>> >> >> >> string, because it’s easier to add yet more defensive
>>> (and
>>> >> >>> >> >> >> generally
>>> >> >>> >> >> >> useless) null checks than it is to ask your clients to
>>> fix
>>> >> >>> >> >> >> their
>>> >> >>> >> >> >> code.
>>> >> >>> >> >> >> Scale
>>> >> >>> >> >> >> this up, and code bloat ensues, as well as wasted
>>> programmer
>>> >> >>> >> >> >> effort
>>> >> >>> >> >> >> that
>>> >> >>> >> >> >> obscures the places where checking for null really does
>>> >> >>> >> >> >> matter.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> In a recent version of Xcode, Apple introduced an
>>> extension
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> C/C++/Objective-C that expresses the nullability of
>>> pointers
>>> >> >>> >> >> >> in
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> type
>>> >> >>> >> >> >> system via new nullability qualifiers . Nullability
>>> >> >>> >> >> >> qualifiers
>>> >> >>> >> >> >> express
>>> >> >>> >> >> >> nullability as part of the declaration of strchr  [2]:
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> __nullable char *strchr(__nonnull const char *s, int c);
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> With this, programmers and tools alike can better reason
>>> >> >>> >> >> >> about
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> use
>>> >> >>> >> >> >> of
>>> >> >>> >> >> >> strchr with null pointers.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> We’d like to contribute the implementation (and there is
>>> a
>>> >> >>> >> >> >> patch
>>> >> >>> >> >> >> attached
>>> >> >>> >> >> >> at the end [3]), but since this is a nontrivial
>>> extension to
>>> >> >>> >> >> >> all
>>> >> >>> >> >> >> of
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> C
>>> >> >>> >> >> >> family of languages that Clang supports, we believe that
>>> it
>>> >> >>> >> >> >> needs to
>>> >> >>> >> >> >> be
>>> >> >>> >> >> >> discussed here first.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Goals
>>> >> >>> >> >> >> We have several specific goals that informed the design
>>> of
>>> >> >>> >> >> >> this
>>> >> >>> >> >> >> feature.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Allow the intended nullability to be expressed on all
>>> >> >>> >> >> >> pointers:
>>> >> >>> >> >> >> Pointers
>>> >> >>> >> >> >> are used throughout library interfaces, and the
>>> nullability
>>> >> >>> >> >> >> of
>>> >> >>> >> >> >> those
>>> >> >>> >> >> >> pointers is an important part of the API contract with
>>> users.
>>> >> >>> >> >> >> It’s
>>> >> >>> >> >> >> too
>>> >> >>> >> >> >> simplistic to only allow function parameters to have
>>> >> >>> >> >> >> nullability,
>>> >> >>> >> >> >> for
>>> >> >>> >> >> >> example, because it’s also important information for data
>>> >> >>> >> >> >> members,
>>> >> >>> >> >> >> pointers-to-pointers (e.g., "a nonnull pointer to a
>>> nullable
>>> >> >>> >> >> >> pointer
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> an
>>> >> >>> >> >> >> integer”), arrays of pointers, etc.
>>> >> >>> >> >> >> Enable better tools support for detecting nullability
>>> >> >>> >> >> >> problems:
>>> >> >>> >> >> >> The
>>> >> >>> >> >> >> nullability annotations should be useful for tools
>>> >> >>> >> >> >> (especially
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> static
>>> >> >>> >> >> >> analyzer) that can reason about the use of null, to give
>>> >> >>> >> >> >> warnings
>>> >> >>> >> >> >> about
>>> >> >>> >> >> >> both
>>> >> >>> >> >> >> missed null checks (the result of strchr could be null…)
>>> as
>>> >> >>> >> >> >> well
>>> >> >>> >> >> >> as
>>> >> >>> >> >> >> for
>>> >> >>> >> >> >> unnecessarily-defensive code.
>>> >> >>> >> >> >> Support workflows where all interfaces provide
>>> nullability
>>> >> >>> >> >> >> annotations:
>>> >> >>> >> >> >> In
>>> >> >>> >> >> >> moving from a world where there are no nullability
>>> >> >>> >> >> >> annotations
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> one
>>> >> >>> >> >> >> where
>>> >> >>> >> >> >> we hope to see many such annotations, we’ve found it
>>> helpful
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> move
>>> >> >>> >> >> >> header-by-header, auditing a complete header to give it
>>> >> >>> >> >> >> nullability
>>> >> >>> >> >> >> qualifiers. Once one has done that, additions to the
>>> header
>>> >> >>> >> >> >> need
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> be
>>> >> >>> >> >> >> held
>>> >> >>> >> >> >> to the same standard, so we need a design that allows us
>>> to
>>> >> >>> >> >> >> warn
>>> >> >>> >> >> >> about
>>> >> >>> >> >> >> pointers that don’t provide nullability annotations for
>>> some
>>> >> >>> >> >> >> declarations in
>>> >> >>> >> >> >> a header that already has some nullability annotations.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Zero effect on ABI or code generation: There are a huge
>>> >> >>> >> >> >> number
>>> >> >>> >> >> >> of
>>> >> >>> >> >> >> interfaces that could benefit from the use of nullability
>>> >> >>> >> >> >> qualifiers,
>>> >> >>> >> >> >> but we
>>> >> >>> >> >> >> won’t get widespread adoption if introducing the
>>> nullability
>>> >> >>> >> >> >> qualifiers
>>> >> >>> >> >> >> means breaking existing code, either in the ABI (say,
>>> because
>>> >> >>> >> >> >> nullability
>>> >> >>> >> >> >> qualifiers are mangled into the type) or at execution
>>> time
>>> >> >>> >> >> >> (e.g.,
>>> >> >>> >> >> >> because a
>>> >> >>> >> >> >> non-null pointer ends up being null along some error
>>> path and
>>> >> >>> >> >> >> causes
>>> >> >>> >> >> >> undefined behavior).
>>> >> >>> >> >> >>
>>> >> >>> >> >> >>
>>> >> >>> >> >> >>
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Why not __attribute__((nonnull))?
>>> >> >>> >> >> >> Clang already has an attribute to express nullability,
>>> >> >>> >> >> >> “nonnull”,
>>> >> >>> >> >> >> which
>>> >> >>> >> >> >> we
>>> >> >>> >> >> >> inherited from GCC [4]. The “nonnull” attribute can be
>>> placed
>>> >> >>> >> >> >> on
>>> >> >>> >> >> >> functions
>>> >> >>> >> >> >> to indicate which parameters cannot be null: one either
>>> >> >>> >> >> >> specifies
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> indices of the arguments that cannot be null, e.g.,
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> extern void *my_memcpy (void *dest, const void *src,
>>> size_t
>>> >> >>> >> >> >> len)
>>> >> >>> >> >> >> __attribute__((nonnull (1, 2)));
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> or omits the list of indices to state that all pointer
>>> >> >>> >> >> >> arguments
>>> >> >>> >> >> >> cannot
>>> >> >>> >> >> >> be
>>> >> >>> >> >> >> null, e.g.,
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> extern void *my_memcpy (void *dest, const void *src,
>>> size_t
>>> >> >>> >> >> >> len)
>>> >> >>> >> >> >> __attribute__((nonnull));
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> More recently, “nonnull”  has grown the ability to be
>>> applied
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> parameters, and one can use the companion attribute
>>> >> >>> >> >> >> returns_nonnull
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> state
>>> >> >>> >> >> >> that a function returns a non-null pointer:
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> extern void *my_memcpy (__attribute__((nonnull)) void
>>> *dest,
>>> >> >>> >> >> >> __attribute__((nonnull)) const void *src, size_t len)
>>> >> >>> >> >> >> __attribute__((returns_nonnull));
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> There are a number of problems here. First, there are
>>> >> >>> >> >> >> different
>>> >> >>> >> >> >> attributes
>>> >> >>> >> >> >> to express the same idea at different places in the
>>> grammar,
>>> >> >>> >> >> >> and
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> use of
>>> >> >>> >> >> >> the “nonnull” attribute on the function actually has an
>>> >> >>> >> >> >> effect
>>> >> >>> >> >> >> on
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> function parameters can get very, very confusing. Quick,
>>> >> >>> >> >> >> which
>>> >> >>> >> >> >> pointers
>>> >> >>> >> >> >> are
>>> >> >>> >> >> >> nullable vs. non-null in this example?
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> __attribute__((nonnull)) void *my_realloc (void *ptr,
>>> size_t
>>> >> >>> >> >> >> size);
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> According to that declaration, ptr is nonnull and the
>>> >> >>> >> >> >> function
>>> >> >>> >> >> >> returns
>>> >> >>> >> >> >> a
>>> >> >>> >> >> >> nullable pointer… but that’s the opposite of how it reads
>>> >> >>> >> >> >> (and
>>> >> >>> >> >> >> behaves,
>>> >> >>> >> >> >> if
>>> >> >>> >> >> >> this is anything like a realloc that cannot fail).
>>> Moreover,
>>> >> >>> >> >> >> because
>>> >> >>> >> >> >> these
>>> >> >>> >> >> >> two attributes are declaration attributes, not type
>>> >> >>> >> >> >> attributes,
>>> >> >>> >> >> >> you
>>> >> >>> >> >> >> cannot
>>> >> >>> >> >> >> express that nullability of the inner pointer in a
>>> >> >>> >> >> >> multi-level
>>> >> >>> >> >> >> pointer
>>> >> >>> >> >> >> or an
>>> >> >>> >> >> >> array of pointers, which makes these attributes verbose,
>>> >> >>> >> >> >> confusing,
>>> >> >>> >> >> >> and
>>> >> >>> >> >> >> not
>>> >> >>> >> >> >> sufficiently generally. These attributes fail the first
>>> of
>>> >> >>> >> >> >> our
>>> >> >>> >> >> >> goals.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> These attributes aren’t as useful as they could be for
>>> tools
>>> >> >>> >> >> >> support
>>> >> >>> >> >> >> (the
>>> >> >>> >> >> >> second and third goals), because they only express the
>>> >> >>> >> >> >> nonnull
>>> >> >>> >> >> >> case,
>>> >> >>> >> >> >> leaving
>>> >> >>> >> >> >> no way to distinguish between the unannotated case
>>> (nobody
>>> >> >>> >> >> >> has
>>> >> >>> >> >> >> documented
>>> >> >>> >> >> >> the nullability of some parameter) and the nullable case
>>> (we
>>> >> >>> >> >> >> know
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> pointer can be null). From a tooling perspective, this
>>> is a
>>> >> >>> >> >> >> killer:
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> static analyzer absolutely cannot warn that one has
>>> forgotten
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> check
>>> >> >>> >> >> >> for
>>> >> >>> >> >> >> null for every unannotated pointer, because the
>>> >> >>> >> >> >> false-positive
>>> >> >>> >> >> >> rate
>>> >> >>> >> >> >> would be
>>> >> >>> >> >> >> astronomical.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Finally, we’ve recently started considering violations
>>> of the
>>> >> >>> >> >> >> __attribute__((nonnull)) contract to be undefined
>>> behavior,
>>> >> >>> >> >> >> which
>>> >> >>> >> >> >> fails
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> last of our goals. This is something we could debate
>>> further
>>> >> >>> >> >> >> if
>>> >> >>> >> >> >> it
>>> >> >>> >> >> >> were
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> only problem, but these declaration attributes fall all
>>> of
>>> >> >>> >> >> >> our
>>> >> >>> >> >> >> criteria, so
>>> >> >>> >> >> >> it’s not worth discussing.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Nullability Qualifiers
>>> >> >>> >> >> >> We propose the addition of a new set of type qualifiers,
>>> >> >>> >> >> >> spelled
>>> >> >>> >> >> >> __nullable, __nonnull, and __null_unspecified, to Clang.
>>> >> >>> >> >> >> These
>>> >> >>> >> >> >> are
>>> >> >>> >> >> >> collectively known as nullability qualifiers and may be
>>> >> >>> >> >> >> written
>>> >> >>> >> >> >> anywhere any
>>> >> >>> >> >> >> other type qualifier may be written (such as const) on
>>> any
>>> >> >>> >> >> >> type
>>> >> >>> >> >> >> subject
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> the following restrictions:
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Two nullability qualifiers shall not appear in the same
>>> set
>>> >> >>> >> >> >> of
>>> >> >>> >> >> >> qualifiers.
>>> >> >>> >> >> >> A nullability qualifier shall qualify any pointer type,
>>> >> >>> >> >> >> including
>>> >> >>> >> >> >> pointers
>>> >> >>> >> >> >> to objects, pointers to functions, C++ pointers to
>>> members,
>>> >> >>> >> >> >> block
>>> >> >>> >> >> >> pointers,
>>> >> >>> >> >> >> and Objective-C object pointers.
>>> >> >>> >> >> >> A nullability qualifier in the declaration-specifiers
>>> applies
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> innermost pointer type of each declarator (e.g.,
>>> __nonnull
>>> >> >>> >> >> >> int *
>>> >> >>> >> >> >> is
>>> >> >>> >> >> >> equivalent to int * __nonnull).
>>> >> >>> >> >> >> A nullability qualifier applied to a typedef of a
>>> >> >>> >> >> >> nullability-qualified
>>> >> >>> >> >> >> pointer type shall specify the same nullability as the
>>> >> >>> >> >> >> underlying
>>> >> >>> >> >> >> type
>>> >> >>> >> >> >> of
>>> >> >>> >> >> >> the typedef.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> The meanings of the three nullability qualifiers are as
>>> >> >>> >> >> >> follows:
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> __nullable: the pointer may store a null value at
>>> runtime (as
>>> >> >>> >> >> >> part
>>> >> >>> >> >> >> of
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> API contract)
>>> >> >>> >> >> >> __nonnull: the pointer should not store a null value at
>>> >> >>> >> >> >> runtime
>>> >> >>> >> >> >> (as
>>> >> >>> >> >> >> part
>>> >> >>> >> >> >> of the API contract). it is possible that the value can
>>> be
>>> >> >>> >> >> >> null,
>>> >> >>> >> >> >> e.g.,
>>> >> >>> >> >> >> in
>>> >> >>> >> >> >> erroneous historic uses of an API, and it is up to the
>>> >> >>> >> >> >> library
>>> >> >>> >> >> >> implementor
>>> >> >>> >> >> >> to decide to what degree she will accommodate such
>>> clients.
>>> >> >>> >> >> >> __null_unspecified: it is unclear whether the pointer
>>> can be
>>> >> >>> >> >> >> null or
>>> >> >>> >> >> >> not.
>>> >> >>> >> >> >> Use of this type qualifier is extremely rare in
>>> practice, but
>>> >> >>> >> >> >> it
>>> >> >>> >> >> >> fills
>>> >> >>> >> >> >> a
>>> >> >>> >> >> >> small but important niche when auditing a particular
>>> header
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> add
>>> >> >>> >> >> >> nullability qualifiers: sometimes the nullability
>>> contract
>>> >> >>> >> >> >> for a
>>> >> >>> >> >> >> few
>>> >> >>> >> >> >> APIs in
>>> >> >>> >> >> >> the header is unclear even when looking at the
>>> implementation
>>> >> >>> >> >> >> for
>>> >> >>> >> >> >> historical
>>> >> >>> >> >> >> reasons, and establishing the contract requires more
>>> >> >>> >> >> >> extensive
>>> >> >>> >> >> >> study.
>>> >> >>> >> >> >> In
>>> >> >>> >> >> >> such cases, it’s often best to mark that pointer as
>>> >> >>> >> >> >> __null_unspecified
>>> >> >>> >> >> >> (which will help silence the warning about unannotated
>>> >> >>> >> >> >> pointers
>>> >> >>> >> >> >> in a
>>> >> >>> >> >> >> header)
>>> >> >>> >> >> >> and move on, coming back to __null_unspecified pointers
>>> when
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> appropriate
>>> >> >>> >> >> >> graybeard has been summoned out of retirement [5].
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Assumes-nonnull Regions
>>> >> >>> >> >> >> We’ve found that it's fairly common for the majority of
>>> >> >>> >> >> >> pointers
>>> >> >>> >> >> >> within
>>> >> >>> >> >> >> a
>>> >> >>> >> >> >> particular header to be __nonnull. Therefore, we’ve
>>> >> >>> >> >> >> introduced
>>> >> >>> >> >> >> assumes-nonnull regions that assume that certain
>>> unannotated
>>> >> >>> >> >> >> pointers
>>> >> >>> >> >> >> implicitly get the __nonnull nullability qualifiers.
>>> >> >>> >> >> >> Assumes-nonnull
>>> >> >>> >> >> >> regions
>>> >> >>> >> >> >> are marked by pragmas:
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> #pragma clang assume_nonnull begin
>>> >> >>> >> >> >>         __nullable char *strchr(const char *s, int c);
>>> // s
>>> >> >>> >> >> >> is
>>> >> >>> >> >> >> inferred
>>> >> >>> >> >> >> to
>>> >> >>> >> >> >> be __nonnull
>>> >> >>> >> >> >> void *my_realloc (__nullable void *ptr, size_t size); //
>>> >> >>> >> >> >> my_realloc
>>> >> >>> >> >> >> is
>>> >> >>> >> >> >> inferred to return __nonnull
>>> >> >>> >> >> >> #pragma clang assume_nonnull end
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> We infer __nonnull within an assumes_nonnull region when:
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> The pointer is a non-typedef declaration, such as a
>>> function
>>> >> >>> >> >> >> parameter,
>>> >> >>> >> >> >> variable, or data member, or the result type of a
>>> function.
>>> >> >>> >> >> >> It’s
>>> >> >>> >> >> >> very
>>> >> >>> >> >> >> rare
>>> >> >>> >> >> >> for one to warn typedefs to specify nullability
>>> information;
>>> >> >>> >> >> >> rather,
>>> >> >>> >> >> >> it’s
>>> >> >>> >> >> >> usually the user of the typedef that needs to specify
>>> >> >>> >> >> >> nullability.
>>> >> >>> >> >> >> The pointer is a single-level pointer, e.g., int* but not
>>> >> >>> >> >> >> int**,
>>> >> >>> >> >> >> because
>>> >> >>> >> >> >> we’ve found that programmers can get confused about the
>>> >> >>> >> >> >> nullability
>>> >> >>> >> >> >> of
>>> >> >>> >> >> >> multi-level pointers (is it a __nullable pointer to
>>> __nonnull
>>> >> >>> >> >> >> pointers,
>>> >> >>> >> >> >> or
>>> >> >>> >> >> >> the other way around?) and inferring nullability for any
>>> of
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> pointers in
>>> >> >>> >> >> >> a multi-level pointer compounds the situation.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Note that no #include may occur within an assumes_nonnull
>>> >> >>> >> >> >> region,
>>> >> >>> >> >> >> and
>>> >> >>> >> >> >> assumes_nonnull regions cannot cross header boundaries.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Type System Impact
>>> >> >>> >> >> >> Nullability qualifiers are mapped to type attributes
>>> within
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> Clang
>>> >> >>> >> >> >> type
>>> >> >>> >> >> >> system, but a nullability-qualified pointer type is not
>>> >> >>> >> >> >> semantically
>>> >> >>> >> >> >> distinct from its unqualified pointer type. Therefore,
>>> one
>>> >> >>> >> >> >> may
>>> >> >>> >> >> >> freely
>>> >> >>> >> >> >> convert between nullability-qualified and
>>> >> >>> >> >> >> non-nullability-qualified
>>> >> >>> >> >> >> pointers, or between nullability-qualified pointers with
>>> >> >>> >> >> >> different
>>> >> >>> >> >> >> nullability qualifiers. One cannot overload on
>>> nullability
>>> >> >>> >> >> >> qualifiers,
>>> >> >>> >> >> >> write
>>> >> >>> >> >> >> C++ class template partial specializations that identify
>>> >> >>> >> >> >> nullability
>>> >> >>> >> >> >> qualifiers, or inspect nullability via type traits in any
>>> >> >>> >> >> >> way.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Said more strongly, removing nullability qualifiers from
>>> a
>>> >> >>> >> >> >> well-formed
>>> >> >>> >> >> >> program will not change its behavior in any way, nor
>>> will the
>>> >> >>> >> >> >> semantics
>>> >> >>> >> >> >> of a
>>> >> >>> >> >> >> program change when any set of (well-formed) nullability
>>> >> >>> >> >> >> qualifiers
>>> >> >>> >> >> >> are
>>> >> >>> >> >> >> added to it. Operationally, this means that nullability
>>> >> >>> >> >> >> qualifiers
>>> >> >>> >> >> >> are
>>> >> >>> >> >> >> not
>>> >> >>> >> >> >> part of the canonical type in Clang’s type system, and
>>> that
>>> >> >>> >> >> >> any
>>> >> >>> >> >> >> warnings we
>>> >> >>> >> >> >> produce based on nullability information will
>>> necessarily be
>>> >> >>> >> >> >> dependent
>>> >> >>> >> >> >> on
>>> >> >>> >> >> >> Clang’s ability to retain type sugar during semantic
>>> >> >>> >> >> >> analysis.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> While it’s somewhat exceptional for us to introduce new
>>> type
>>> >> >>> >> >> >> qualifiers
>>> >> >>> >> >> >> that don’t produce semantically distinct types, we feel
>>> that
>>> >> >>> >> >> >> this is
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> only plausible design and implementation strategy for
>>> this
>>> >> >>> >> >> >> feature:
>>> >> >>> >> >> >> pushing
>>> >> >>> >> >> >> nullability qualifiers into the type system semantically
>>> >> >>> >> >> >> would
>>> >> >>> >> >> >> cause
>>> >> >>> >> >> >> significant changes to the language (e.g., overloading,
>>> >> >>> >> >> >> partial
>>> >> >>> >> >> >> specialization) and break ABI (due to name mangling) that
>>> >> >>> >> >> >> would
>>> >> >>> >> >> >> drastically
>>> >> >>> >> >> >> reduce the number of potential users, and we feel that
>>> >> >>> >> >> >> Clang’s
>>> >> >>> >> >> >> support
>>> >> >>> >> >> >> for
>>> >> >>> >> >> >> maintaining type sugar throughout semantic analysis is
>>> >> >>> >> >> >> generally
>>> >> >>> >> >> >> good
>>> >> >>> >> >> >> enough
>>> >> >>> >> >> >> [6] to get the benefits of nullability annotations in our
>>> >> >>> >> >> >> tools.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> Looking forward to our discussion.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> - Doug (with Jordan Rose and Anna Zaks)
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> [1]
>>> >> >>> >> >> >>
>>> >> >>> >> >> >>
>>> >> >>> >> >> >>
>>> http://en.wikipedia.org/wiki/Tony_Hoare#Apologies_and_retractions
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__en.wikipedia.org_wiki_Tony-5FHoare-23Apologies-5Fand-5Fretractions&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=CnzuN65ENJ1H9py9XLiRvC_UQz6u3oG6GUNn7_wosSM&m=bljRv73Hdesdao5bJgcUiiE_nylTGrbpSZs4E6AE8Fo&s=_PIx9QaKZlsYZF-JpRLnkzkjNK6U-ip_DqRWv3Xrhcw&e=>
>>> >> >>> >> >> >> [2] The standard description of strchr seems to imply
>>> that
>>> >> >>> >> >> >> the
>>> >> >>> >> >> >> parameter
>>> >> >>> >> >> >> cannot be null
>>> >> >>> >> >> >> [3] The patch is complete, but should be reviewed on
>>> >> >>> >> >> >> cfe-commits
>>> >> >>> >> >> >> rather
>>> >> >>> >> >> >> than here. There are also several logic parts to this
>>> >> >>> >> >> >> monolithic
>>> >> >>> >> >> >> patch:
>>> >> >>> >> >> >> (a) __nonnull/__nullable/__null_unspecified type
>>> specifiers
>>> >> >>> >> >> >> (b) nonnull/nullable/null_unspecified syntactic sugar for
>>> >> >>> >> >> >> Objective-C
>>> >> >>> >> >> >> (c) Warning about inconsistent application of nullability
>>> >> >>> >> >> >> specifiers
>>> >> >>> >> >> >> within a given header
>>> >> >>> >> >> >> (d) assume_nonnnull begin/end pragmas
>>> >> >>> >> >> >> (e) Objective-C null_resettable property attribute
>>> >> >>> >> >> >> [4]
>>> >> >>> >> >> >>
>>> https://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__gcc.gnu.org_onlinedocs_gcc_Function-2DAttributes.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=CnzuN65ENJ1H9py9XLiRvC_UQz6u3oG6GUNn7_wosSM&m=bljRv73Hdesdao5bJgcUiiE_nylTGrbpSZs4E6AE8Fo&s=7XK1ONRGcffCS5T_y1QAAPCMo3IPe50NIVPvlF08JmM&e=>
>>> >> >>> >> >> >> (search
>>> >> >>> >> >> >> for “nonnull”)
>>> >> >>> >> >> >> [5] No graybeards were harmed in the making of this
>>> feature.
>>> >> >>> >> >> >> [6] Template instantiation is the notable exception here,
>>> >> >>> >> >> >> because it
>>> >> >>> >> >> >> always canonicalizes types.
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> <nullability.patch>
>>> >> >>> >> >> >> _______________________________________________
>>> >> >>> >> >> >> cfe-dev mailing list
>>> >> >>> >> >> >> cfe-dev at cs.uiuc.edu
>>> >> >>> >> >> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>> >> >>> >> >> >>
>>> >> >>> >> >> >>
>>> >> >>> >> >> >>
>>> >> >>> >> >> >> _______________________________________________
>>> >> >>> >> >> >> cfe-dev mailing list
>>> >> >>> >> >> >> cfe-dev at cs.uiuc.edu
>>> >> >>> >> >> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>> >> >>> >> >> >>
>>> >> >>> >> >> >
>>> >> >>> >> >> >
>>> >> >>> >> >> > _______________________________________________
>>> >> >>> >> >> > cfe-dev mailing list
>>> >> >>> >> >> > cfe-dev at cs.uiuc.edu
>>> >> >>> >> >> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>> >> >>> >> >> >
>>> >> >>> >> >
>>> >> >>> >> >
>>> >> >>> >
>>> >> >>> >
>>> >> >>
>>> >> >>
>>> >> >
>>> >> >
>>> >> > _______________________________________________
>>> >> > cfe-dev mailing list
>>> >> > cfe-dev at cs.uiuc.edu
>>> >> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>> >> >
>>> >
>>> >
>>>
>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150629/f955506e/attachment.html>