[llvm] r249490 - InstCombine: Fold comparisons between unguessable allocas and other pointers

[Hans Wennborg via llvm-commits]

Hi Nuno,

My understanding from the discussion before the patch is that while it
would be undefined behaviour to dereference or do pointer arithmetic
with invalid pointers, comparing for equality is valid and should
compare the underlying representations.

In this example:

  int f(int *p) {
    int x;
    return p == &x;
  }

we can't fold the comparison to undef, because it must return 0 unless
the caller guesses the value of &x (which my commit argues we can act
as-if is impossible).

 - Hans


On Wed, Oct 7, 2015 at 3:05 PM, Nuno Lopes via llvm-commits
<llvm-commits at lists.llvm.org> wrote:
> Hi,
>
> Sorry, I'm a bit late.
> I was wondering why can't fold these comparisons with undef?
>
> AFAIK, the C standard says that producing a pointer (one element) past the
> end of the buffer is undefined behavior, and therefore it's undefined
> behavior to guess addresses.  So I think we can be more aggressive and
> assume pointer guesses can take any value (and therefore the comparison can
> be both true and false -- like we do with uninitialized memory loads).
>
> Nuno
>
> -----Original Message----- From: Hans Wennborg via llvm-commits
> Sent: Wednesday, October 7, 2015 1:20 AM
> To: llvm-commits at lists.llvm.org
> Subject: [llvm] r249490 - InstCombine: Fold comparisons between unguessable
> allocas and other pointers
>
>
> Author: hans
> Date: Tue Oct  6 19:20:07 2015
> New Revision: 249490
>
> URL: http://llvm.org/viewvc/llvm-project?rev=249490&view=rev
> Log:
> InstCombine: Fold comparisons between unguessable allocas and other pointers
>
> This will allow us to optimize code such as:
>
>  int f(int *p) {
>    int x;
>    return p == &x;
>  }
>
> as well as:
>
>  int *allocate(void);
>  int f() {
>    int x;
>    int *p = allocate();
>    return p == &x;
>  }
>
> The folding can only be done under certain circumstances. Even though p and
> &x
> cannot alias, the comparison must still return true if the pointer
> representations are equal. If a user successfully generates a p that's a
> correct guess for &x, comparison should return true even though p is an
> invalid
> pointer.
>
> This patch argues that if the address of the alloca isn't observable outside
> the
> function, the function can act as-if the address is impossible to guess from
> the
> outside. The tricky part is keeping the act consistent: if we fold p == &x
> to
> false in one place, we must make sure to fold any other comparisons based on
> those pointers similarly. To ensure that, we only fold when &x is involved
> exactly once in comparison instructions.
>
> Differential Revision: http://reviews.llvm.org/D13358