[flang-commits] [clang] [flang] [llvm] [openmp] [OpenMP][Offload] Add offload runtime support for dyn_groupprivate clause (PR #152831)

[Alex Duran via flang-commits]

================
@@ -515,8 +524,60 @@ Error GenericKernelTy::launch(GenericDeviceTy &GenericDevice, void **ArgPtrs,
   llvm::SmallVector<void *, 16> Args;
   llvm::SmallVector<void *, 16> Ptrs;
 
+  uint32_t NumThreads[3] = {KernelArgs.ThreadLimit[0],
+                            KernelArgs.ThreadLimit[1],
+                            KernelArgs.ThreadLimit[2]};
+  uint32_t NumBlocks[3] = {KernelArgs.NumTeams[0], KernelArgs.NumTeams[1],
+                           KernelArgs.NumTeams[2]};
+  if (!isBareMode()) {
+    NumThreads[0] = getNumThreads(GenericDevice, NumThreads);
+    NumBlocks[0] = getNumBlocks(GenericDevice, NumBlocks, KernelArgs.Tripcount,
+                                NumThreads[0], KernelArgs.ThreadLimit[0] > 0);
+  }
+
+  uint32_t MaxBlockMemSize = GenericDevice.getMaxBlockSharedMemSize();
+  uint32_t DynBlockMemSize = KernelArgs.DynCGroupMem;
+  uint32_t TotalBlockMemSize = StaticBlockMemSize + DynBlockMemSize;
+  if (StaticBlockMemSize > MaxBlockMemSize)
+    return Plugin::error(ErrorCode::INVALID_ARGUMENT,
+                         "Static block memory size exceeds maximum");
+  else if (static_cast<DynCGroupMemFallbackType>(
+               KernelArgs.Flags.DynCGroupMemFallback) ==
+               DynCGroupMemFallbackType::Abort &&
+           TotalBlockMemSize > MaxBlockMemSize)
+    return Plugin::error(
+        ErrorCode::INVALID_ARGUMENT,
+        "Static and dynamic block memory size exceeds maximum");
+
+  void *DynBlockMemFbPtr = nullptr;
+  uint32_t DynBlockMemLaunchSize = DynBlockMemSize;
+
+  DynCGroupMemFallbackType DynBlockMemFb = DynCGroupMemFallbackType::None;
+  if (DynBlockMemSize && (!GenericDevice.hasNativeBlockSharedMem() ||
----------------
adurang wrote:

Let me try to explain myself a bit better. I was thinking of two cases that I don't feel are currently captured:

a) The plugin might use shared memory that is not statically determined for a given kernel (e.g. for example we use reduction buffers that might be of different sizes for the same kernel depending on the invocation). This kind of memory doesn't seem to be accounted in the TotalBlockMemSize.

b) If multiple kernels are queued and can run concurrently on a device, even if TotalBlockMemSizeK1 (of Kernel 1) and TotalBlockMemSizeK2 are < MaxBlockMemSize, I think TotalBlockMemSizeK1 + TotalBlockMemSizeK2  can be > MaxBlockMemSize which will probably fail.

https://github.com/llvm/llvm-project/pull/152831