[llvm-dev] Do I have to lock ThreadSafeContext and ThreadSafeModule in single threaded env?

[Lang Hames via llvm-dev]

Hi Mo,

I’m new to JIT and just started experiencing on some example code with
> ORCv2, but could not get my head around with the thread safety.


The short version is that ThreadSafeContext is a pair of an LLVMContext and
a mutex, and that mutex is locked whenever you call
ThreadSafeModule::withModuleDo on a module using that ThreadSafeContext.

That means that two Modules that share a Context can't be accessed
simultaneously (when wrapped in and accessed through ThreadSafeModule and
ThreadSafeContext).

In this system you rarely have to do anything explicitly. The idea is that
you access the module via withModuleDo and that takes care of the locking
for you.

Scenario 1, Application has multiple threads and each thread spawns its own
> JIT instance, within each JIT instance, there are multiple
> ThreadSafeModules share one ThreadSafeContext
> Do I have to lock TSM/ TSC?


You should always use withModuleDo to access the module, regardless of your
setup. This prevents you from writing non-thread-safe code that might be
difficult to debug if you add threading in later. What's important here is
knowing the locking implications of the access pattern, since they'll
affect how much parallelism you can get out of your JIT.

In your Scenario 1 each thread accesses a single LLVMContext. There are no
LLVMContexts shared between threads, so no chance for lock contention.
That's good.

 Scenario 2,  If JIT itself has multiple compilation threads, like via
> setting NumCompileThreads in LLJIT classes, and each ThreadSafeModule has
> its own ThreadSafeContext, so context is not shared


In this scenario each JIT has multiple threads, but each thread is
operating on a single module with its own LLVMContext. Again there is no
chance for lock contention, so no barrier to parallelism.

The only scenario where locking becomes relevant is when you have multiple
threads and a single ThreadSafeContext shared between two or more
ThreadSafeModules. Consider a JIT with 10 threads and 10 llvm::Modules all
sharing a single ThreadSafeContext. If you issue a lookup that triggers
materialization of all 10 modules simultaneously then they'll be sent to
each of your 10 worker threads to be compiled, but as soon as the 1st
worker thread takes the context lock the other 9 threads will be blocked
from progressing. Eventually the 1st worker thread will relinquish the lock
and the next worker thread will take it, preventing the other 8 threads
from progressing, and so on. You have 10 threads, but only one thread at a
time can actually do any work -- the single LLVMContext has serialized
compilation.

My recommendation is to always start with one context per module, and only
share contexts if you see memory consumption from the contexts becoming an
issue.

-- Lang.

On Thu, Jul 8, 2021 at 12:13 AM mo xiaoming via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi all,
>
> I’m new to JIT and just started experiencing on some example code with
> ORCv2, but could not get my head around with the thread safety.
>
> Scenario 1, Application has multiple threads and each thread spawns its
> own JIT instance, within each JIT instance, there are multiple
> ThreadSafeModules share one ThreadSafeContext
>
> Do I have to lock TSM/ TSC?
>
> Scenario 2,  If JIT itself has multiple compilation threads, like via
> setting NumCompileThreads in LLJIT classes, and each ThreadSafeModule has
> its own ThreadSafeContext, so context is not shared
>
> In this case, do I have to lock? If so, what’s the granularity of each
> lock should be, anything before IR generation in one locking?
>
>
> I think in scenario 1, locking is not needed because of no concurrent
> access. And I believe locking is a mandatory in scenario 2, but I don’t
> know why...
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210730/11fe81a2/attachment.html>