[all-commits] [llvm/llvm-project] 785636: [flang][cuda] duplicate host_device procedures ear...

[Zhen Wang via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 78563625bdce600d1a4af2905ed42c17ff881a94
      https://github.com/llvm/llvm-project/commit/78563625bdce600d1a4af2905ed42c17ff881a94
  Author: Zhen Wang <zhenw at nvidia.com>
  Date:   2026-09-21 (Mon, 21 Sep 2026)

  Changed paths:
    M flang/include/flang/Optimizer/Builder/CUFCommon.h
    M flang/include/flang/Optimizer/Dialect/CUF/Attributes/CUFAttr.h
    M flang/include/flang/Optimizer/Support/Utils.h
    M flang/include/flang/Optimizer/Transforms/Passes.td
    M flang/lib/Lower/CallInterface.cpp
    M flang/lib/Optimizer/Builder/CUFCommon.cpp
    M flang/lib/Optimizer/Support/Utils.cpp
    M flang/lib/Optimizer/Transforms/CMakeLists.txt
    M flang/lib/Optimizer/Transforms/CUDA/CUFDeviceFuncTransform.cpp
    M flang/lib/Optimizer/Transforms/CUDA/CUFDeviceGlobal.cpp
    A flang/lib/Optimizer/Transforms/CUDA/CUFDuplicateDeviceFunc.cpp
    M flang/lib/Optimizer/Transforms/CUDA/CUFPredefinedVarToGPU.cpp
    M flang/test/Fir/CUDA/cuda-device-func-transform-aio.mlir
    A flang/test/Fir/CUDA/cuda-device-func-transform-copies.mlir
    A flang/test/Fir/CUDA/cuda-duplicate-device-func.mlir

  Log Message:
  -----------
  [flang][cuda] duplicate host_device procedures early and outline the copies under their original name (#224400)

An `attributes(host,device)` procedure is lowered as a single
`func.func`, which `cuf-transform-device-func` clones into the GPU
module at the end of the FIR pipeline. By then the optimizer has already
placed the one body under the host policy, so with `-fstack-arrays` an
automatic array stays a `fir.alloca` and overflows the device stack at
run time:

```fortran
attributes(host,device) integer function sumauto(n)
  integer :: auto(n)
  ...
end function

attributes(global) subroutine kern(a, n)
  a(1) = sumauto(n)          ! device call
end subroutine
```

Procedures reached from device code only through such a procedure, and
device bodies inlined into a kernel, have the same problem.

This adds `cuf-duplicate-device-func`, meant to run before the FIR
optimizer. Every `host_device` procedure, and every procedure without a
device attribute that device code reaches through calls or procedure
references, gets a device copy in the host module, and device code is
redirected to it while host code keeps the original:

```mlir
func.func @sumauto(...) attributes {cuf.proc_attr = #cuf.cuda_proc<host_device>} {
  %auto = fir.alloca !fir.array<?xi32>, %n        // host copy, unchanged
}
func.func @sumauto.device(...) attributes {cuf.proc_attr = #cuf.cuda_proc<device>,
    cuf.device_copy_of = @sumauto,
    fir.allocation_policy = #fir.allocation_policy<stack_arrays = false, ...>} {
  %auto = fir.alloca !fir.array<?xi32>, %n        // placed to the heap by the optimizer
}
func.func @kern(...) attributes {cuf.proc_attr = #cuf.cuda_proc<global>} {
  fir.call @sumauto.device(...)                    // was @sumauto
}
```

The optimizer then treats the two as ordinary functions with their own
policies, and inlining a copy into a kernel carries the heap allocation
along. Declarations are not copied, so runtime calls keep their real
names and the descriptor I/O check in `cuf-transform-device-func` still
fires.

`cuf-transform-device-func` recognizes the `cuf.device_copy_of` marker,
outlines the copy under the original name, since device symbol names are
the cross-unit ABI, restores the original names inside outlined code and
`cuf.kernel` regions, leaves the `host_device` original in the host
module and erases the copy:

```mlir
func.func @sumauto(...) { %auto = fir.alloca ... }         // host module
gpu.module @cuda_device_mod {
  gpu.func @sumauto(...) { %auto = fir.allocmem ... }      // the copy, renamed back
  gpu.func @kern(...) kernel { fir.call @sumauto(...) }
}
```

Without markers the pass behaves as before, so it is correct in either
pipeline order.

The device code gathering is factored out of `cuf-transform-device-func`
into `cuf::collectDeviceCode`, shared by both passes.
`fir::getPresentedFunction` and `fir::getPresentedCallee` let
diagnostics and remarks report a copy as the procedure it copies, and
`fir::getPresentableFunctionName` uses them, so a remark inside
`@sumauto.device` names `sumauto`. Lowering and the copies share
`cuf::setDeviceAllocationPolicy`.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications