[llvm-dev] asan for allocas on powerpc64

[Jay Foad via llvm-dev]

>>     Currently test/asan/TestCases/alloca_vla_interact.cc is XFAILed for
>>     powerpc64. I've had a look at why it doesn't work. I think the only
>>     problem is in the call to __asan_allocas_unpoison that is inserted at
>>     the end of the "for" loop (just before a stackrestore instruction).
>>
>>     The call function is created something like this (paraphrasing from
>>     lib/Transfoms/Instrumentation/AddressSanitizer.cpp):
>>
>>       // call __asan_allocas_unpoison(uptr top, uptr bottom);
>>       // NB "top" here means lowest address and "bottom" means highest!
>>
>>       IRB.CreateCall(
>>         AsanAllocasUnpoisonFunc,
>>         {
>>           IRB.CreateLoad(DynamicAllocaLayout),
>>     IRB.CreatePointerToInt(SaveRestoreInst->getOperand(0), IntptrTy)
>>         }
>>       );
>>
>>     I think the problem is that the operand to stackrestore is the new
>>     native sp register value to restore, and this code is assuming that
>>     that will be a higher address than all the allocas that are being
>>     unallocated. But on PowerPC64, the native sp is always lower than the
>>     address of the most recent alloca by MaxCallFrameSize bytes, to leave
>>     space for outgoing call arguments. So I think the second argument to
>>     __asan_allocas_unpoison needs to be SaveRestoreInst->getOperand(0) +
>>     MaxCallFrameSize, but I don't know how to implement that.
>>
>>     Thoughts?
>>
>
> Yeah, you are right, we rely on stackrestore to get "bottom" parameter for
> __asan_allocas_unpoison and indeed PowerPC64 seems to be special here.
> However, right now I don't see a suitable way how to get MaxCallFrameSize
> from LLVM backend in ASan pass, investigation is in progress.

One idea I had for a solution was to change llvm.stacksave so that it
*always* returned native sp + MaxCallFrameSize, and do the
corresponding adjustment in llvm.stackrestore. In most cases this
would just replace a mov instruction with an addi instruction.

I don't know if this would cause other problems, or if it would be
acceptable to the target maintainers.

Jay.