[llvm-dev] Custom calling convention & ARM target

[Alexander Mitin via llvm-dev]

Hi.

> What normally happens is that the call gets marked as clobbering LR
> which triggers ARMFrameLowering.cpp to save it in the prologue and
> restore it in the epilogue.
Do you mean the callee should save/restore LR?
If so, then this is not as good as it could be.
Let me clarify things a bit more.
We don't want it to be like that:
handlerFunc0:  ; our CC
   push lr
   ; do smth
   pop lr
   b handlerFuncX ; branch to the next handler

We need it like this:
starterFunc:   ; cdecl or whatever standard CC
  mov lr, offset of label1
  push lr                   ; or store lr somewhere else
  bl handlerFuncX   ; or even just b handlerFuncX
label1:
 ; do something on return
 ; ..
terminatorFunc:   ; our CC
  pop lr          ; or reload lr from some other memory
  bx lr

Also this code would look good:
handlerFunc0:  ; our CC
   ; do smth
   push lr
   bl someFuncWithStandardCC
   pop lr
   ; do smth else
   b handlerFuncX ; branch to the next handler

I wasn't expecting that it would work out-of-the-box and I don't think
it's a bug in LLVM :)
I'm just looking for guidance on the best way to implement it.

> You mention above that you've patterned your changes on the GHC
> convention, which suppresses prologue and epilogue. That's probably
> where I'd start to look for the problem. But don't trust what's there
> already: I'm not sure how functional the GHC convention is on ARM, it
> seems like it could only work if it guaranteed *every* call was a tail
> call.
For our implementation we don't suppress prologue/epilogue. However,
no callee-saved registers allowed to reduce mem-reg operations.
This CC is very specific and it was never intended to be used widely.
And you are correct that it would only work if it guaranteed every
call was a tail call - and that's the idea how the interpreter of
our VA Smalltalk virtual machine works. It's much faster than any
implementation in C language.

Kind Regards,
Alexander Mitin