[llvm-dev] Fwd: asan for allocas on powerpc64

[Jay Foad via llvm-dev]

(Resending with the correct mailing list address.)

Hi,

Currently test/asan/TestCases/alloca_vla_interact.cc is XFAILed for
powerpc64. I've had a look at why it doesn't work. I think the only
problem is in the call to __asan_allocas_unpoison that is inserted at
the end of the "for" loop (just before a stackrestore instruction).

The call function is created something like this (paraphrasing from
lib/Transfoms/Instrumentation/AddressSanitizer.cpp):

  // call __asan_allocas_unpoison(uptr top, uptr bottom);
  // NB "top" here means lowest address and "bottom" means highest!

  IRB.CreateCall(
    AsanAllocasUnpoisonFunc,
    {
      IRB.CreateLoad(DynamicAllocaLayout),
      IRB.CreatePointerToInt(SaveRestoreInst->getOperand(0), IntptrTy)
    }
  );

I think the problem is that the operand to stackrestore is the new
native sp register value to restore, and this code is assuming that
that will be a higher address than all the allocas that are being
unallocated. But on PowerPC64, the native sp is always lower than the
address of the most recent alloca by MaxCallFrameSize bytes, to leave
space for outgoing call arguments. So I think the second argument to
__asan_allocas_unpoison needs to be SaveRestoreInst->getOperand(0) +
MaxCallFrameSize, but I don't know how to implement that.

Thoughts?

Thanks,
Jay.