[PATCH] D99642: For non-null pointer checks, do not descend through out-of-bounds GEPs

[Nuno Lopes via Phabricator via llvm-commits]

nlopes added a comment.

In D99642#2660961 <https://reviews.llvm.org/D99642#2660961>, @fhahn wrote:

> In D99642#2660952 <https://reviews.llvm.org/D99642#2660952>, @nlopes wrote:
>
>> In D99642#2660864 <https://reviews.llvm.org/D99642#2660864>, @lebedev.ri wrote:
>>
>>> In D99642#2660849 <https://reviews.llvm.org/D99642#2660849>, @fhahn wrote:
>>>
>>>> In D99642#2660824 <https://reviews.llvm.org/D99642#2660824>, @lebedev.ri wrote:
>>>>
>>>>> alive2 again doesn't agree that non-inbounds GEP is allowed to produce null pointer: https://alive2.llvm.org/ce/z/9wfL5x
>>>>
>>>> Interesting, but also surprising, especially because the LangRef explicitly calls out GEPs without `inbounds` to wrap silently?
>>>
>>> Yes.
>>> I think it's another example of alive2 trying to invent a memory model that isn't fit for the real world.
>>
>> Let me try to explain our reasoning:
>>
>> - GEP doesn't change the object of a pointer, just the offset within that object. Having this rule enables a lot of optimizations (e.g. you know "for free" that `gep %p, %x` and `gep %q, %y` can't alias if %p/%q are the result of e.g. distinct alloca/malloc, even if you know nothing about %x/%y).
>
> I think that's fine for AA, which generally looks at pointers that are dereferenced. It seems a bit unfortunate though that we determine that the pointer is based on object but still may not be de-referenceable if the object is though.

That case only happens if some form of UB happens (e.g., gep inbounds that goes OOB, passing a pointer as argument with some incorrect attribute, etc).
The warning in the alias FAQ is because of pointer replacement AFAIR. Just because 2 pointers compare equal, you can't replace one with the other blindly, because one might be dereferenceable and the other not (because one overflowed inbounds, for example).
Pointer comparisons are just tricky.

>> - Pointer comparisons need to account for the previous rule plus how we lower pointer comparisons (an integer comparison in assembly). For pointers whose offsets are inbounds, integer comparison == object/offset comparison. If one of the pointers is OOB, then things get complicated because in assembly an OOB pointer may be equal to an inbounds pointer, while in IR we would rather not have that situation to enable optimizations. This lead us to pursue a semantics where the result of a comparison with an OOB ptr is non-deterministic, such that the IR can assume it's always false and assembly can sometimes produce true. The implementation in Alive2 is not complete FYI.
>
> I think we need the capability to express arbitrary pointer arithmetic without the assumption that it cannot point to potentially a different object, e.g. for runtime checks that a pointer does not wrap or go out-of-bounds of a certain object. An answer could of course be that you have to use `ptrtoint/inttoptr` for that case, but I think in practice this is not what is used today for such cases. (and not what the LangRef suggests, at least as long as the pointer is not dereferenced).

That's true. I'm aware of the loop vectorizer that inserts runtime checks to only dispatch to the vectorized loop if some pointers don't alias at runtime. We probably don't want to use ptr2int for this.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D99642/new/

https://reviews.llvm.org/D99642