[llvm-dev] Custom calling convention & ARM target

[Alexander Mitin via llvm-dev]

Hello.

For our project needs we implemented a custom calling convention. The
main goals are to pass function arguments in registers and always use
tailcall optimization for calls to functions with our CC when
applicable.
Function arguments are always pointers and the maximum number of
arguments is 5. No frame pointer register is in use for this CC.  No varargs.
Finally, there are not any callee-saving registers.
This approach worked successfully for x86 arch.

For ARM we are having troubles with the LR register.
The problem is that when there is a return from a function using our CC
the existing LLVM machinery emits 'mov pc, lr' instruction which looks fine.
The expectation is that we would return to a function which called our
CC function for the first time (no tailcall for the first time call).
But at the moment the return LR register contains an invalid value
because is wasn't preserved.
So the question is how to preserve LR register in the best way? My
current idea is to write a MachineFunctionPass which would add LR
register spill instruction to stack or some other memory and add LR
reload instruction on return.

Does this seem like a reasonable approach?  Thank you for your
time and consideration.

Kind Regards,

Alexander Mitin