[all-commits] [llvm/llvm-project] a10ca8: [offload] use argument pointer array in olLaunchKe...

[Piotr Balcer via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: a10ca8ed6454361b3889a9d5b41b97d065984648
      https://github.com/llvm/llvm-project/commit/a10ca8ed6454361b3889a9d5b41b97d065984648
  Author: Piotr Balcer <piotr.balcer at intel.com>
  Date:   2026-05-29 (Fri, 29 May 2026)

  Changed paths:
    M llvm/include/llvm/Frontend/Offloading/Utility.h
    M llvm/lib/Frontend/Offloading/Utility.cpp
    M llvm/tools/llvm-gpu-loader/llvm-gpu-loader.cpp
    M llvm/tools/llvm-gpu-loader/llvm-gpu-loader.h
    M offload/include/Shared/APITypes.h
    M offload/liboffload/API/Kernel.td
    M offload/liboffload/src/OffloadImpl.cpp
    M offload/plugins-nextgen/amdgpu/src/rtl.cpp
    M offload/plugins-nextgen/common/src/PluginInterface.cpp
    M offload/plugins-nextgen/cuda/src/rtl.cpp
    M offload/plugins-nextgen/level_zero/dynamic_l0/L0DynWrapper.cpp
    M offload/plugins-nextgen/level_zero/dynamic_l0/level_zero/ze_api.h
    M offload/plugins-nextgen/level_zero/include/L0Kernel.h
    M offload/plugins-nextgen/level_zero/src/L0Kernel.cpp
    M offload/unittests/Conformance/include/mathtest/DeviceContext.hpp
    M offload/unittests/Conformance/include/mathtest/Support.hpp
    M offload/unittests/Conformance/lib/DeviceContext.cpp
    M offload/unittests/OffloadAPI/device_code/CMakeLists.txt
    A offload/unittests/OffloadAPI/device_code/composite.cpp
    M offload/unittests/OffloadAPI/device_code/multiargs.cpp
    M offload/unittests/OffloadAPI/event/olGetEventElapsedTime.cpp
    M offload/unittests/OffloadAPI/kernel/olLaunchKernel.cpp
    M offload/unittests/OffloadAPI/kernel/olLaunchKernelCooperative.cpp
    M offload/unittests/OffloadAPI/memory/olMemcpy.cpp
    M offload/unittests/OffloadAPI/queue/olLaunchHostFunction.cpp
    M offload/unittests/OffloadAPI/queue/olWaitEvents.cpp

  Log Message:
  -----------
  [offload] use argument pointer array in olLaunchKernel (#194333)

This PR changes olLaunchKernel to accept an array of pointers to
arguments:
```
  void *ArgPtrs[] = {&A, &B, &C};
  size_t ArgSizes[] = {sizeof(A), sizeof(B), sizeof(C)};

  olLaunchKernel(Queue, Device, Kernel, &LaunchArgs, std::size(ArgPtrs), ArgPtrs, ArgSizes);
```

The newly proposed interface is implementable by existing and
anticipated
backends, is familiar to CUDA programmers, eliminates the extraneous
construction of a contiguous arguments buffer, replacing it with
constructing
an array of pointers, sidesteps the alignment requirements, does not
require reading program image metadata where it's impractical, and
enables
a compliant SYCL implementation to be built on top of it.

The ArgSizes array is required to support OpenCL, which does not have
native support for launching a kernel with an argument pointer array, or
a reliable way of retrieving argument sizes for a kernel.

## Mapping the proposed API to backends

CUDA and Level-Zero both support accepting an array of pointers to
kernel arguments, through `cuLaunchKernel`

(https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__EXEC.html#group__CUDA__EXEC_1gb8f3dc3031b40da29d5f9a7139e52e15)
and `zeCommandListAppendLaunchKernelWithArguments`

(https://oneapi-src.github.io/level-zero-spec/level-zero/latest/core/api.html#zecommandlistappendlaunchkernelwitharguments)
respectively.

For OpenCL, which requires the kernel arguments to be set separately
from the kernel launch, a potential implementation can extract the
number of arguments from the OpenCL API, and then iterate over the
argument pointer and size arrays:
```
  cl_uint num_args = 0;
  cl_int err = clGetKernelInfo(kernel, CL_KERNEL_NUM_ARGS,
                               sizeof(num_args), &num_args, NULL);
  for (cl_uint i = 0; i < num_args; ++i) {
    clSetKernelArg(kernel, i, ArgSizes[i], ArgPtrs[i]);
  }
```
The AMDGPU plugin needs to construct a contiguous buffer using the array
of argument pointers. To do this conversion, we need to have offsets
at which to place the arguments. Here, luckily, as mentioned before,
the AMD plugin already reads kernel metadata from the program image.
The implementation simply retrieves size and offset for each argument
from the kernel image, and then uses it to populate a buffer.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications