[llvm] [Offload] Move RPC server handling to a dedicated thread (PR #112988)

[Joseph Huber via llvm-commits]

jhuber6 wrote:

> Creating extra thread can be tricky because there are cases that people don't like extra thread created. For example, QMCPack disables hidden helper thread due to this reason. Some search online shows that `CUDA_LAUNCH_BLOCKING` is actually a debug feature. IMHO we can just claim that this is not supported (TBH we don't have to support things prefix with either `CUDA` or `HIP`), but users can still choose to use at their discretion.

It's a debug feature, but AFAIK it's still against the programming model because they expect things like that to be possible. So we're basically relying on implicit behavior that's not guaranteed by their documentation. Even without that, this still doesn't work in an async scenario without helper threads. I'm also not a fan of user threads, which is why I elected to make this one go to sleep when it's not needed so it's at least not wasting resources. I think right now this will always create the thread, since I figured it's fine if it just goes to sleep immediately and is never woken up by a kernel launch that needs an RPC server, but I could make it only do that if we need the RPC server at all, then it's a pay for what you use scenario.

https://github.com/llvm/llvm-project/pull/112988