[llvm] [Offload][AMDGPU] Add env var to control device-to-device memory access (PR #215385)

[Joseph Huber via llvm-commits]

================
@@ -4514,10 +4522,12 @@ Expected<void *> AMDGPUDeviceTy::allocate(size_t Size, void *,
   if (auto Err = MemoryPool->allocate(Size, &Alloc, Alignment))
     return std::move(Err);
 
-  if (Alloc) {
+  if (Alloc && (Kind == TARGET_ALLOC_HOST || Kind == TARGET_ALLOC_SHARED ||
----------------
jhuber6 wrote:

We'd probably need to coordinate if this is something that the OpenMP standard requires or something, but in general I agree that by default these allocations should not list peer access. this is only really a problem in OpenMP because the language doesn't really give you a way to request this, so it's mostly up to if there's any language in the standard for this. At a minimum, this is a behavior that should be moved out of the plugin itself and more into a config, I remember some discussion around that.

I'd say we should flip the default and then expose a function in the plugin that allows peer access, then we choose what default OpenMP should set it to, maybe contingent on an environment variable.

https://github.com/llvm/llvm-project/pull/215385