[LLVMdev] LLVM JIT in multi-threaded environments

[Eric]

Hi all,

I have some questions on some strange behavior I'm seeing with LLVM JIT in a
multithreaded application. If this is the wrong forum for questions like
this, I apologize, and please redirect me.

The setup of the application is a singleton instance of a static Compiler
object, which maintains local copies of an LLVM Module, JIT ExecutionEngine,
and FunctionPassManager with registered optimization passes to be applied to
all functions.

There are a handful of worker threads that pass state pointers to IR
building routines within the Compiler object. These Compiler routines return
a list of llvm::Function pointers to all functions generated for the given
state. These functions can get pretty big, so in the interest of performance
during the optimization passes, I break them up into several smaller
"chunks" that are called by a parent function. Each of these chunks are
setup with InternalLinkage, and the parent is setup with ExternalLinkage.
The parent and all resulting children chunks are passed back to the caller
in the list of llvm::Function pointers. The llvm::Function pointers are
stored in the caller threads.

This list is then run through JIT by calling getPointerToFunction() for each
element. All llvm::Function pointers are kept around for deallocation later.
Each thread client uses these generated functions for a while, then frees
them all by calling freeMachineCodeForFunction() and eraseFromParent() on
each llvm::Function* it allocated. Each worker thread then allocates another
set of functions and JITs them, continuing indefinitely for the life of the
application.

All of the thread interaction around LLVM is protected by locks. Only 1
thread can be constructing a function and calling the JIT at any given time.
The same is true for freeing the function.

Now, the problem. The application runs indefinitely with a single client
thread constructing and destructing LLVM functions. Since this is true, I'm
sure there's no memory leaking in the executable 16MB machine code buffer
from forgetting to deallocate functions. As soon as a 2nd thread gets
involved, however, the application runs for a few hundred iterations and
then I get a JIT: ran out of space for machine code error from LLVM.

At first I would suspect some bad thread interaction in my client code,
however I've expanded the scope of locks so that only 1 thread can talk to
the LLVM singleton at a time. I have spent considerable effort ensuring this
is true.

So my question is, is there anything inside the LLVM JITEmitter that could
be thread aware? What if multiple threads are JIT'ing functions to the same
module with the same basic block names (presumably these would get renamed
automatically?).

At this point I am at a loss to understand how the LLVM JIT could be acting
any differently with multiple threads calling JIT routines, than in a
single-threaded environment, unless the identical IR build routines
executing for multiple threads are creating some sort of conflicting state
within the list of internal FunctionBlocks or something of that nature. Any
and all ideas would be appreciated.

Thanks much,
Eric Yew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090227/06763838/attachment.html>