[LLVMdev] Tail call optimization thoughts

[Arnold Schwaighofer]

Implementing tail call opt could look like the following:

0.)a fast calling convention (maybe use the current  
CallingConv::Fast, or create a CallingConv::TailCall)
1.) lowering of formal arguments
   like for example x86_LowerCCCArguments in  stdcall mode
   we need to make sure that later mentioned CALL_CLOBBERED_REG is  
not used (remove it from available
   registers in callingconvention for argument passing?)
2.)lowering of the call:
   *if it can be shown that call really is tail call (next  
instruction is a return):
      -move the arguments to the correct position on the stack
      -create a REALTAILCALL SelDAG node holding
        tailcallee : if the tailcallee is dynamic, not a  
TargetGlobalAddress or the like,
        lower a move CALL_CLOBBERED_REG tailcallee instruction else  
attach a
        TargetGlobalAddress or the like
      -the size of the stack adjustment
      the realtailcall would be a no op operation that makes sure  
that epilogue is
      lowered before it (looks like a call), remember that we have a  
real tail call
      (like setBytesToPopOnReturn() , a setIsRealTailCall()) in the  
targetlowering())
   *else emit regular function call with same calling conventions
3.)lower of the return:
   *if we are dealing with a realtailcall (getIsRealTailCall)
       look for REALTAILCALL use its operands to
       emit a TC_RETURN node holding stacksize, tailcallee
       the tc_return node would again be a pseudoop
   *else lower normal return


When generating the epilog the two operands of the the tc_return  
machine instruction are used to emit code that adjust the  
stackpointer and jumps to the tailcallee (either label or register).  
Like it is done for EH_RETURN.

in X86RegisterInfo.cpp we would then have
TargetRegisterInfo::emitEpilogue() {
   ...
   if (RetOpcode== X86::TC_RETURN){
     if (isDynamicCallee(RetOpCode))
       add esp  {stack adjustment from tc_return}
       jmp {register operand of tc_return}
     } else
       add esp  {stack adjustment from tc_return}
       jmp {targetfunction operand}
     }
}

resulting code for dynamic function
	
   mov  ecx  esi  #load callee, say esi holds the address of the  
tailcalled
   epilogue
   #TAILCALL
   add esp 8  #caller has 2 more arg
   jmp ecx

if the targetfunction is known

   epilogue
   #TAILCALL
   add esp 8 #caller has 2 more arg
   jmp _targetfunction

Should also work for architectures other than x86.

any critique, comments welcome
regards arnold