[llvm] [Offload] Allow allocations to be freed without knowing their type (PR #140889)

[Joseph Huber via llvm-commits]

================
@@ -639,6 +636,44 @@ struct CUDADeviceTy : public GenericDeviceTy {
     return OFFLOAD_SUCCESS;
   }
 
+  int free_non_blocking(void *TgtPtr) override {
----------------
jhuber6 wrote:

I don't know how to handle this honestly, the problem is that the CUDA API thought it was a good idea to tie memory allocations with the default stream so it can be blocked by kernels executing. I'm wondering if we should just have a spare stream we use for allocations and only use async. This was mostly a hack that's not even used anymore. Hopefully later I can get rid of this after I'm doing finishing `malloc` in `libc`.

https://github.com/llvm/llvm-project/pull/140889