[llvm-dev] RFC: alloca -- specify address space for allocation

[Sanjoy Das via llvm-dev]

Some vocabulary, to avoid confusion later:

 - the "alloca" == the pointer to the stack slot that the alloca
   allocated (e.g. %rbp - 0x30)

 - the "value" stored or contained in an alloca == at a given point
   during the execution, what is the N-byte value that is present in
   the location that is the result of the alloca instruction.

This is my understanding of the situation: the allocas in question are
not themselves GC pointers, but the value stored in those allocas are.
We could represent the alloca's as address space 0 pointers if we had
a way to do a precise liveness analysis on them late.

However, I think there is a basic problem with doing a precise late
liveness analysis to determine which allocas live over a safepoint
*contain* a gc reference.  Since the pointee type of a pointer is not
a strong distinction (more so once we finally have a singular "ptr"
pointer type) we can have IR like:

 entry:
  br i1 %cond, label %left, label %right

 left:
  %x = alloca addrspace(1)*
  store addrspace(1)* %gcptr, addrspace(1)** %x
  %xc = bitcast addrspace(1)** %x to i8*
  br label %merge

 right:
  %y = alloca i64
  store i64 42, i64* %y
  %yc = bitcast i64* %y to i8*
  br label %merge

 merge:
  %m = phi i8* [ %xc ] [ %yc ]
  safepoint()
  br i1 %cond, label %left1, label %right1

 right1:
  ret void

 left1:
  %xd = bitcast i8* %m to addrspace(1)**
  %gcptr = load addrspace(1)*, addrspace(1)** %xd
  use(%gcptr)

Now the only thing live over the safepoint is the phi of allocas ==
`%m` but depending on control flow the runtime value of `%m` (which
will be an alloca) can either contain a legitimate gc pointer, or
i64 42.

Therefore, if we wish to precisely distinguish, at compile time,
between "normal" allocas that cannot contains gc pointers and
"interesting" allocas that *can contain* gc pointers, then we need to
make the pointer types of the "normal" and "interesting" allocas
fundamentally different (i.e. be of different address spaces).
However, I think re-using addresspace(1) for this is a mistake, we
should use a second address space for this (addressspace(2) let's say)
that distinguish pointers that point to locations that may *contain*
gc pointers, but are not themselves gc pointers.

-- Sanjoy