[llvm-dev] Potential issue with noalias @malloc and @realloc

[Sanjoy Das via llvm-dev]

Hi all,

I think I've spotted a semantic issue with marking @malloc and
@realloc as noalias.  Say we have the program:

int f() {
  int* p0 = malloc(size of(int));
  free(p0);
  int* p1 = malloc(sizeof(int));
  if (!p1) return 20;
  int value = 0;
  for (int i = 0; i < 1; i++) {
    *p1 = 20;
    value = *p1;
    if (false)  // "false" is obscured in a way the compiler can't fathom
      if (p0 == p1)
        a();
      else
        b();
  }
  return result;
}

The program above is well defined, and will always return 20.
However, if we first unswitch loops:

int f() {
  int* p0 = malloc(size of(int));
  free(p0);
  int* p1 = malloc(sizeof(int));
  if (!p1) return 20;
  int value = 0;
  if (p0 == p1) {
    for (int i = 0; i < 1; i++) {
      *p1 = 20;
      value = *p1;
      if (false)
        a();
    }
  } else {
    // Other copy of the loop that calls b() in dead code
  }
  return result;
}

and then run GVN::propgateEquality that replaces one use of p1 with p0
but not the other (for some reason, say the other use was obscured
behind a function call):

int f() {
  int* p0 = malloc(size of(int));
  free(p0);
  int* p1 = malloc(sizeof(int));
  if (!p1) return 20;
  int value = 0;
  if (p0 == p1) {
    for (int i = 0; i < 1; i++) {
      *p0 = 20;     // S0
      value = *p1;  // L0
      if (false)
        a();
    }
  } else {
    // Other copy of the loop that calls b() in dead code
  }
  return result;
}


Now we have a problem -- since p0 NoAlias p1 (distinct @malloc()
calls), we can reorder S0 and L0 (or LICM L0):


int f() {
  int* p0 = malloc(size of(int));
  free(p0);
  int* p1 = malloc(sizeof(int));
  if (!p1) return 20;
  int value = 0;
  if (p0 == p1) {
    for (int i = 0; i < 1; i++) {
      value = *p1;  // L0
      *p0 = 20;     // S0
      if (false)
        a();
    }
  } else {
    // Other copy of the loop that calls b() in dead code
  }
  return result;
}

and we'll now return garbage if p0 == p1 (likely) and p1 is not null
(also likely).

I do not yet have a solution for this.  Let me know what you think!

Thanks,
-- Sanjoy