[LLVMdev] Proposal: stack/context switching within a thread

[Kenneth Uildriks]

On Sat, Apr 10, 2010 at 12:54 PM, Jeffrey Yasskin <jyasskin at google.com> wrote:
> I took the liberty of forwarding this to the Stackless Python list,
> since they switch stacks, and I got a response at
> http://thread.gmane.org/gmane.comp.python.stackless/4464/focus=4467.
> The upshot is that they really need the ability to allocate only a
> tiny amount of space for each thread and grow that as the thread
> actually uses more stack. The way they accomplish that now is by
> copying the entire stack to the heap on a context switch, and having
> all threads share the main C stack. This isn't quite as bad as it
> sounds because it only happens to threads that call into C extension
> modules. Pure Python threads operate entirely within heap Python
> frames. Still, it would be nice to support this use case.
>
> Kenneth, I don't want to insist that the first version of this be both
> a floor wax _and_ a dessert topping, but is there a natural extension
> to supporting what Stackless needs that we could add later? It looks
> like swapcontext() simply repoints the stack pointer and restores the
> registers, while Stackless wants it to be able to allocate memory and
> copy the stack. Maybe that implies a "mode" argument?
>
> Alternately, Stackless could probably work with a segmented stack
> mechanism like Ian Taylor implemented in gcc for Go. Do you see
> anything that would prevent us from layering segmented stacks on top
> of this context switching mechanism later?
>
> Thanks,
> Jeffrey

As I see it, the context switching mechanism itself needs to know
where to point the stack register when switching.  The C routines take
an initial stack pointer when creating the context, and keep track of
it from there.  If we don't actually need to interoperate with
contexts created from the C routines, we have a lot more freedom.

Anyway, one approach would be to expose intrinsics to interrogate an
inactive context, to get its initial stack pointer (the one it was
created with) and its current stack pointer, and also  to modify both
before making the context active again.

I don't see any reason why this scheme wouldn't also be compatible
with segmented stacks.  In fact, one could segment an inactive context
stack at any time rather than copying the whole thing, as long as you
can assume that there aren't any pointers into the context's stack
living outside the context.