[llvm-dev] Reducing the number of ptrtoint/inttoptrs that are generated by LLVM

[Juneyoung Lee via llvm-dev]

If the integer casted from pointer is compared, GVN may introduce a problem:

int y[10];
int x[40];
intptr_t iy = (intptr_t)&y[0];
intptr_t ix = (intptr_t)&x[40];
int *x2 = (int*)ix;

if ((psub inbounds &x[40], x2) == 0) {
  if (ix == iy) {
    *(int *)iy = 100; // initially, this has no UB
    // => *(int *)ix = 100; // by ix == iy
    // => *x2 = 100; // by x2 == (int*)ix
    // => x[40] = 100; // by psub; raises UB
  }
}

In this example, the program initially has no UB, but after optimizations
it raises UB.
psub should not replace x2 with x[40] here.

Juneyoung Lee

On Wed, Jan 23, 2019 at 6:33 AM Sanjoy Das <sanjoy at playingwithpointers.com>
wrote:

> On Tue, Jan 22, 2019 at 11:07 AM Juneyoung Lee
> <juneyoung.lee at sf.snu.ac.kr> wrote:
> > Ralf pointed out that psub cannot be used for propagating pointer
> equality if pointer-cast integer is involved;
> >
> > a = p
> > b = inttoptr(ptrtoint p)
> > if ((psub inbounds a b) == 0) {
> > use(b) // replacing b with a may be problematic, as it is essentially
> folding inttoptr(ptrtoint p) -> p, which is discussed at
> https://bugs.llvm.org/show_bug.cgi?id=34548
> > }
> >
> > I'm sorry for the confusion. To propagate pointer equality, we certainly
> need a better solution. :\
>
> I don't immediately see the problem.  Firstly because we branch on
> (psub a b), a and b must have a common provenance.  If `a` is an
> interior pointer to this common allocation then we're fine.  So the
> only case where the psub will be == 0 and `a` will not be
> dereferenceable is if a points to one past the end of some allocation.
>
> The most obvious "full" example I can come up in this setting is:
>
> // Assume the stack layout is x followed by y
> int y[10];
> int x[40];
> int* a = &x[40]
> int* b = inttoptr(ptrtoint a)
> if ((psub inbounds a b) == 0) {
>   *b = 9; // Sets y[0] = 9
> }
>
> But this code is problematic for other reasons (so I think it has to
> be UB) -- if we allow assigning to y[0] like above then this breaks
> alias analysis on alloca's right? `y` above is not escaped, so we
> should be able to assume that nothing writes to it.
>
> -- Sanjoy
>


-- 

Juneyoung Lee
Software Foundation Lab, Seoul National University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190123/d6000802/attachment.html>