[llvm] [Offload] Make olLaunchKernel test thread safe (PR #149497)

[Ross Brunton via llvm-commits]

================
@@ -642,17 +643,20 @@ struct CUDADeviceTy : public GenericDeviceTy {
   }
 
   /// Synchronize current thread with the pending operations on the async info.
-  Error synchronizeImpl(__tgt_async_info &AsyncInfo) override {
+  Error synchronizeImpl(__tgt_async_info &AsyncInfo,
+                        bool RemoveQueue) override {
     CUstream Stream = reinterpret_cast<CUstream>(AsyncInfo.Queue);
     CUresult Res;
     Res = cuStreamSynchronize(Stream);
 
     // Once the stream is synchronized, return it to stream pool and reset
     // AsyncInfo. This is to make sure the synchronization only works for its
     // own tasks.
-    AsyncInfo.Queue = nullptr;
-    if (auto Err = CUDAStreamManager.returnResource(Stream))
-      return Err;
+    if (RemoveQueue) {
+      AsyncInfo.Queue = nullptr;
----------------
RossBrunton wrote:

When the device is de-inited, all streams in the stream manager are deinited and dropped.

For liboffload specifically, since devices are not cleared, this happens during the final liboffload `olShutDown` call.

https://github.com/llvm/llvm-project/pull/149497