[flang-commits] [flang] f96feba - [flang][cuda] Share the managed companion pointer across translation units (#225710)
via flang-commits
flang-commits at lists.llvm.org
Thu Sep 24 03:49:19 PDT 2026
Author: Kareem Ergawy
Date: 2026-09-24T12:49:14+02:00
New Revision: f96febaf2701fc5f1979a4e0b49bdd97cae3fdb6
URL: https://github.com/llvm/llvm-project/commit/f96febaf2701fc5f1979a4e0b49bdd97cae3fdb6
DIFF: https://github.com/llvm/llvm-project/commit/f96febaf2701fc5f1979a4e0b49bdd97cae3fdb6.diff
LOG: [flang][cuda] Share the managed companion pointer across translation units (#225710)
A non-allocatable managed module variable is accessed through a
companion pointer in the `__nv_managed_data__` section, which the CUDA
runtime fills in with the unified memory address at module
initialization. Every translation unit referencing the variable emitted
its own internal pointer and registered it. The runtime populates only
the registration performed first for a given variable name (*), so every
other unit was left loading through a null pointer and crashed at run
time.
Emit a single pointer with external linkage instead. The unit defining
the variable emits the definition, zero-initialized and in
`__nv_managed_data__`, and registers it; a unit that only USEs the
variable emits a declaration and no registration, resolving to that
definition at link time.
(*) I verified this using the following reproducer:
`exp_mod.f90`:
```fortran
module m
implicit none
type :: t
integer :: n
real, allocatable, managed :: a(:)
end type
type(t) :: g
end module
```
`exp_use.f90`:
```fortran
program p
use m
allocate(g%a(4))
g%n = 4
print *, "n=", g%n, " size=", size(g%a)
end program
```
And the following sequence of events:
1. Compile each source individually to its `*.o` file.
2. Manually link the 2 object files: 2.1. If we link `exp_mod.o
exp_use.o` (object file that does actually use the global last), then we
get a seg fault because the companion global is not initialized. 2.2. If
we reverse the linking order `exp_use.o exp_mod.o`, the companion
pointer of the users object file get proper initialized and the binary
does not seg fault.
If we add uses to both files, then we get a seg fault regradless of the
linking order.
Added:
flang/test/Fir/CUDA/cuda-managed-pointer-linkage.cuf
Modified:
flang/lib/Optimizer/Transforms/CUDA/CUFAddConstructor.cpp
flang/test/Fir/CUDA/cuda-constructor-2.f90
flang/test/Fir/CUDA/cuda-managed-descriptor-component.fir
Removed:
################################################################################
diff --git a/flang/lib/Optimizer/Transforms/CUDA/CUFAddConstructor.cpp b/flang/lib/Optimizer/Transforms/CUDA/CUFAddConstructor.cpp
index ee51703b62caf..3acc1de16c638 100644
--- a/flang/lib/Optimizer/Transforms/CUDA/CUFAddConstructor.cpp
+++ b/flang/lib/Optimizer/Transforms/CUDA/CUFAddConstructor.cpp
@@ -83,12 +83,23 @@ static constexpr llvm::StringRef cudaFortranCtorName{
static constexpr llvm::StringRef managedPtrSuffix{".managed.ptr"};
static constexpr llvm::StringRef cudaCompiledSymbolName{"Mcuda_compiled"};
-/// Create an 8-byte pointer global in the __nv_managed_data__ section.
-/// The CUDA runtime populates this pointer with the unified memory address
-/// when the module is initialized via __cudaInitModule.
+/// Create the 8-byte companion pointer global holding the unified memory
+/// address of \p globalOp, which the CUDA runtime populates when the module is
+/// initialized via __cudaInitModule.
+///
+/// The pointer has external linkage so that the whole program shares a single
+/// one. Only the translation unit that defines the variable emits a definition
+/// (zero-initialized, in the __nv_managed_data__ section, and registered);
+/// every other translation unit emits a declaration and resolves to that same
+/// definition at link time.
+///
+/// A pointer per translation unit does not work: the runtime populates only
+/// the registration it performs first for a given variable name, so any other
+/// unit would be left loading through a null pointer.
static fir::GlobalOp createManagedPointerGlobal(fir::FirOpBuilder &builder,
mlir::ModuleOp mod,
- fir::GlobalOp globalOp) {
+ fir::GlobalOp globalOp,
+ bool isDefinition) {
mlir::MLIRContext *ctx = mod.getContext();
std::string ptrGlobalName = (globalOp.getSymName() + managedPtrSuffix).str();
auto ptrTy = fir::LLVMPointerType::get(ctx, mlir::IntegerType::get(ctx, 8));
@@ -100,7 +111,13 @@ static fir::GlobalOp createManagedPointerGlobal(fir::FirOpBuilder &builder,
auto ptrGlobal = fir::GlobalOp::create(
builder, globalOp.getLoc(), ptrGlobalName, /*isConstant=*/false,
/*isTarget=*/false, ptrTy, initAttr,
- /*linkage=*/builder.createInternalLinkage());
+ /*linkage=*/builder.createExternalLinkage());
+
+ // Leaving the region empty makes this a declaration of the definition
+ // emitted by the defining translation unit.
+ if (!isDefinition)
+ return ptrGlobal;
+
ptrGlobal.setSectionAttr(builder.getStringAttr("__nv_managed_data__"));
mlir::Region ®ion = ptrGlobal.getRegion();
@@ -262,8 +279,8 @@ static bool hasRegisteredGlobals(mlir::ModuleOp mod,
}
if (!gpuSymTable.lookup(globalOp.getSymName()))
continue;
- // Non-allocatable managed globals register a companion pointer local to
- // this translation unit, so they register even when defined elsewhere.
+ // Non-allocatable managed globals still need a companion pointer
+ // declaration when defined elsewhere, so they count here too.
if (attr.getValue() == cuf::DataAttribute::Managed &&
!mlir::isa<fir::BaseBoxType>(globalOp.getType()))
return true;
@@ -431,9 +448,9 @@ struct CUFAddConstructor
attr.getValue() == cuf::DataAttribute::Managed &&
!mlir::isa<fir::BaseBoxType>(globalOp.getType());
- // Non-allocatable managed globals register a companion pointer local
- // to this translation unit, so they register even when defined
- // elsewhere.
+ // Non-allocatable managed globals are still visited when defined
+ // elsewhere: such a unit emits no registration, but it does need a
+ // declaration of the companion pointer to load through.
if (!definesGlobal(globalOp) && !isNonAllocManagedGlobal)
continue;
@@ -442,12 +459,18 @@ struct CUFAddConstructor
case cuf::DataAttribute::Constant:
case cuf::DataAttribute::Managed: {
if (isNonAllocManagedGlobal) {
- hasNonAllocManagedGlobal = true;
// Non-allocatable managed globals use pointer indirection:
// a companion pointer in __nv_managed_data__ holds the unified
// memory address, registered via __cudaRegisterManagedVar.
- fir::GlobalOp ptrGlobal =
- createManagedPointerGlobal(builder, mod, globalOp);
+ // The pointer is shared across the program, so it is defined and
+ // registered only by the unit that defines the variable; other
+ // units just declare it.
+ bool definesVariable = definesGlobal(globalOp);
+ fir::GlobalOp ptrGlobal = createManagedPointerGlobal(
+ builder, mod, globalOp, definesVariable);
+ if (!definesVariable)
+ break;
+ hasNonAllocManagedGlobal = true;
auto func = fir::runtime::getRuntimeFunc<mkRTKey(
CUFRegisterManagedVariable)>(loc, builder);
emitCUFRegistrationCall(builder, loc, idxTy, *dl, kindMap,
diff --git a/flang/test/Fir/CUDA/cuda-constructor-2.f90 b/flang/test/Fir/CUDA/cuda-constructor-2.f90
index 10bee539b5f3e..e97ad80cededc 100644
--- a/flang/test/Fir/CUDA/cuda-constructor-2.f90
+++ b/flang/test/Fir/CUDA/cuda-constructor-2.f90
@@ -146,8 +146,9 @@ module attributes {dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<!llvm.ptr, dense<
}
}
-// Pointer global should be created with section attribute.
-// CHECK: fir.global internal @_QMtestEmanx.managed.ptr <{section = "__nv_managed_data__"}> : !fir.llvm_ptr<i8>
+// The defining unit emits the pointer as an external definition with the
+// section attribute, so the whole program shares a single pointer.
+// CHECK: fir.global external @_QMtestEmanx.managed.ptr <{section = "__nv_managed_data__"}> : !fir.llvm_ptr<i8>
// CHECK: fir.zero_bits !fir.llvm_ptr<i8>
// Constructor should register with CUFRegisterManagedVariable then init module.
@@ -444,3 +445,40 @@ module attributes {dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<!llvm.ptr, dense<
// MARKER: llvm.load volatile
// CHECK: llvm.return
+
+// -----
+
+// A unit that only USEs a non-allocatable managed variable declares the
+// companion pointer and does not register it. The runtime populates only the
+// first registration made for a given variable name, so a pointer per unit
+// would leave every unit but one loading through null.
+//
+// Fortran source:
+// subroutine sub
+// use test ! test declares integer*4, managed :: manx(100)
+// manx(1) = 1
+// end subroutine
+
+module attributes {dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<!llvm.ptr, dense<64> : vector<4xi64>>, #dlti.dl_entry<i64, dense<64> : vector<2xi64>>, #dlti.dl_entry<i32, dense<32> : vector<2xi64>>, #dlti.dl_entry<i8, dense<8> : vector<2xi64>>, #dlti.dl_entry<i1, dense<8> : vector<2xi64>>, #dlti.dl_entry<f64, dense<64> : vector<2xi64>>, #dlti.dl_entry<f32, dense<32> : vector<2xi64>>, #dlti.dl_entry<"dlti.endianness", "little">, #dlti.dl_entry<"dlti.stack_alignment", 128 : i64>>, fir.defaultkind = "a1c4d8i4l4r4", fir.kindmap = "", gpu.container_module, llvm.data_layout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128", llvm.target_triple = "x86_64-unknown-linux-gnu"} {
+
+ // No body: the variable is defined by another translation unit.
+ fir.global @_QMuseronlyEmanx {data_attr = #cuf.cuda<managed>} : !fir.array<100xi32>
+
+ gpu.module @cuda_device_mod {
+ gpu.func @_QMuseronlyPkernel() kernel {
+ gpu.return
+ }
+ fir.global @_QMuseronlyEmanx {data_attr = #cuf.cuda<managed>} : !fir.array<100xi32>
+ }
+}
+
+// The companion pointer is a declaration: external linkage, no section, and no
+// initializer body.
+// CHECK: fir.global external @_QMuseronlyEmanx.managed.ptr : !fir.llvm_ptr<i8>
+// CHECK-NOT: __nv_managed_data__
+// CHECK-NOT: fir.zero_bits !fir.llvm_ptr<i8>
+
+// Neither the registration nor the module init is emitted here.
+// CHECK: llvm.func internal @__cudaFortranConstructor()
+// CHECK-NOT: fir.call @_FortranACUFRegisterManagedVariable
+// CHECK-NOT: fir.call @_FortranACUFInitModule
diff --git a/flang/test/Fir/CUDA/cuda-managed-descriptor-component.fir b/flang/test/Fir/CUDA/cuda-managed-descriptor-component.fir
index 46613be9bed5b..f6e703a462e8e 100644
--- a/flang/test/Fir/CUDA/cuda-managed-descriptor-component.fir
+++ b/flang/test/Fir/CUDA/cuda-managed-descriptor-component.fir
@@ -4,7 +4,12 @@
// descriptor (allocatable/pointer) component instead of aborting.
module attributes {dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<!llvm.ptr, dense<64> : vector<4xi64>>, #dlti.dl_entry<!llvm.ptr<271>, dense<32> : vector<4xi64>>, #dlti.dl_entry<!llvm.ptr<270>, dense<32> : vector<4xi64>>, #dlti.dl_entry<f128, dense<128> : vector<2xi64>>, #dlti.dl_entry<f64, dense<64> : vector<2xi64>>, #dlti.dl_entry<f80, dense<128> : vector<2xi64>>, #dlti.dl_entry<f16, dense<16> : vector<2xi64>>, #dlti.dl_entry<i32, dense<32> : vector<2xi64>>, #dlti.dl_entry<i16, dense<16> : vector<2xi64>>, #dlti.dl_entry<i128, dense<128> : vector<2xi64>>, #dlti.dl_entry<i8, dense<8> : vector<2xi64>>, #dlti.dl_entry<!llvm.ptr<272>, dense<64> : vector<4xi64>>, #dlti.dl_entry<i64, dense<64> : vector<2xi64>>, #dlti.dl_entry<i1, dense<8> : vector<2xi64>>, #dlti.dl_entry<"dlti.endianness", "little">, #dlti.dl_entry<"dlti.stack_alignment", 128 : i64>>, fir.defaultkind = "a1c4d8i4l4r4", fir.kindmap = "", gpu.container_module, llvm.data_layout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128", llvm.target_triple = "x86_64-unknown-linux-gnu"} {
- fir.global @_QMmEobj {data_attr = #cuf.cuda<managed>} : !fir.array<10x!fir.type<_QMmTt{nodl:!fir.box<!fir.heap<!fir.array<?xi32>>>}>>
+ // Initialized: this is the unit defining the variable, which is the one
+ // that registers the companion pointer.
+ fir.global @_QMmEobj {data_attr = #cuf.cuda<managed>} : !fir.array<10x!fir.type<_QMmTt{nodl:!fir.box<!fir.heap<!fir.array<?xi32>>>}>> {
+ %0 = fir.zero_bits !fir.array<10x!fir.type<_QMmTt{nodl:!fir.box<!fir.heap<!fir.array<?xi32>>>}>>
+ fir.has_value %0 : !fir.array<10x!fir.type<_QMmTt{nodl:!fir.box<!fir.heap<!fir.array<?xi32>>>}>>
+ }
gpu.module @cuda_device_mod {
fir.global @_QMmEobj {data_attr = #cuf.cuda<managed>} : !fir.array<10x!fir.type<_QMmTt{nodl:!fir.box<!fir.heap<!fir.array<?xi32>>>}>>
diff --git a/flang/test/Fir/CUDA/cuda-managed-pointer-linkage.cuf b/flang/test/Fir/CUDA/cuda-managed-pointer-linkage.cuf
new file mode 100644
index 0000000000000..ec474bf92d803
--- /dev/null
+++ b/flang/test/Fir/CUDA/cuda-managed-pointer-linkage.cuf
@@ -0,0 +1,53 @@
+! A non-allocatable managed module variable is accessed through a companion
+! pointer in __nv_managed_data__. That pointer is shared by the whole program:
+! the unit defining the variable emits the definition and registers it, and a
+! unit that only USEs the variable emits a declaration and no registration.
+!
+! A pointer per translation unit does not work. The runtime populates only the
+! registration it performs first for a given variable name, so every other unit
+! would be left loading through a null pointer. There is no runtime query to
+! fall back on either: a managed variable has no device symbol, so asking for
+! its address fails with cudaErrorInvalidSymbol.
+
+! The runs share a directory so that the second one picks up the module file
+! written by the first.
+! RUN: rm -rf %t && split-file %s %t
+! RUN: cd %t && bbc -fcuda -gpu=managed -emit-fir defining.cuf -o - \
+! RUN: | fir-opt --cuf-device-global --cuf-add-constructor \
+! RUN: | FileCheck %s --check-prefix=DEFINING
+! RUN: cd %t && bbc -fcuda -gpu=managed -emit-fir useonly.cuf -o - \
+! RUN: | fir-opt --cuf-device-global --cuf-add-constructor \
+! RUN: | FileCheck %s --check-prefix=USEONLY
+
+!--- defining.cuf
+module m
+ integer, managed :: g(100)
+end module
+
+! The defining unit owns the pointer: external linkage, in the
+! __nv_managed_data__ section, with an initializer.
+! DEFINING: fir.global external @_QMmEg.managed.ptr <{section = "__nv_managed_data__"}> : !fir.llvm_ptr<i8> {
+! DEFINING: fir.zero_bits !fir.llvm_ptr<i8>
+
+! It is the unit that registers the pointer and initializes the module.
+! DEFINING: llvm.func internal @__cudaFortranConstructor()
+! DEFINING: fir.address_of(@_QMmEg.managed.ptr) : !fir.ref<!fir.llvm_ptr<i8>>
+! DEFINING: fir.call @_FortranACUFRegisterManagedVariable
+! DEFINING: fir.call @_FortranACUFInitModule
+
+!--- useonly.cuf
+subroutine sub()
+ use m
+ g(1) = 1
+end subroutine
+
+! A unit that only USEs the variable declares the pointer: external linkage,
+! no section, and no initializer, so it resolves to the definition above.
+! USEONLY: fir.global external @_QMmEg.managed.ptr : !fir.llvm_ptr<i8>
+! USEONLY-NOT: __nv_managed_data__
+! USEONLY-NOT: fir.zero_bits !fir.llvm_ptr<i8>
+
+! Registering here too would leave this unit loading through a null pointer.
+! USEONLY: llvm.func internal @__cudaFortranConstructor()
+! USEONLY-NOT: fir.call @_FortranACUFRegisterManagedVariable
+! USEONLY-NOT: fir.call @_FortranACUFInitModule
More information about the flang-commits
mailing list