[LLVMdev] Injecting code before function prolog

[Kenneth Uildriks]

On Wed, Apr 7, 2010 at 12:43 PM, Arlen Cox <arlencox at gmail.com> wrote:
> I'm trying to implement something similar to this:
> http://gcc.gnu.org/wiki/SplitStacks in LLVM.  The reason I want this
> is so that I can have dynamically growing and shrinking stacks in my
> programming language.  In order to do this, I need to be able to check
> for overflow of a stack frame.  The methods of doing this are outlined
> in the link above, but my intention is to pass the current stack limit
> as the first argument to the function.
>
> What I'm hoping to do is to be able to inject the following code (in
> x86 asm, c calling convention) on entry to each function:
> _foo:
>  lea -frame_size(%esp), %eax
>  cmpl %eax, 4(%esp)
>  jb function_entry
>  // handle overflow
> function_entry:
>  function prolog
>  ...
>
> The problem I'm encountering is how to force this before the prolog.
> I'm attempting to add a machine function pass after the emit
> prolog/epilog pass that injects this code, but directly injecting x86
> code seems to be very messy as I have to figure out how LLVM encodes
> the addressing modes and instructions specific to x86.  Additionally,
> directly inject x86 code produces an LLVM that is not target
> independent anymore.
>
> Is there a better way to do this?  Can I maintain target independence?
>  All I really need is to be able to access the stack pointer.
>
> Thanks for your help,
> Arlen Cox

I wonder if it might be better to inject something at callsites. That
way you don't have to copy part of the preexisting stack... just
adjust the stack pointer and then lay down the arguments and return
address.  (Of course you'd have to be able to get to your own stack
variables while you're doing this somehow).  Also, this makes all
issues regarding calling conventions, combining split-stack functions
and non-split-stack functions, and so forth go away.