[flang-commits] [flang] [mlir] [flang][OpenMP] Support allocate clauses on scope constructs (PR #221229)
Sairudra More via flang-commits
flang-commits at lists.llvm.org
Fri Sep 4 07:17:02 PDT 2026
https://github.com/Saieiei created https://github.com/llvm/llvm-project/pull/221229
Flang already lowers OpenMP 5.2 `allocate` clauses on `scope`
constructs to `omp.scope`, but LLVM IR translation previously rejected
them. This change maps the allocation operands to their corresponding
private or firstprivate slots and reuses the existing allocator-backed
private-storage and OpenMPIRBuilder allocation/free paths.
On `omp.scope`, support is currently limited to host-side fixed-size
intrinsic scalar private and firstprivate variables using omitted,
default, or explicit allocators, including `align`. Arrays, descriptors,
pointers, allocatables, derived types, SAVE/common-block storage, and
target/device allocation remain unsupported and explicitly diagnosed.
Assisted-by: GitHub Copilot
>From 06890a56e6a723e0f243dc0914fa231915f23a02 Mon Sep 17 00:00:00 2001
From: Sairudra More <sairudra60 at gmail.com>
Date: Fri, 4 Sep 2026 08:45:01 -0500
Subject: [PATCH] [flang][OpenMP] Support allocate clauses on scope constructs
Flang already lowers OpenMP 5.2 allocate clauses on scope constructs
to omp.scope, but LLVM IR translation previously rejected them.
Map allocate operands to their corresponding private or firstprivate
slots and reuse the existing allocator-backed private-storage path and
OpenMPIRBuilder allocation and cleanup operations. Allocator-backed
storage is created on each dynamic scope entry and released after
privatizer cleanup.
Support is currently limited to host-side fixed-size intrinsic scalar
private and firstprivate variables. Unsupported entities and
target/device allocation continue to be diagnosed.
---
flang/docs/OpenMPSupport.md | 2 +-
flang/lib/Lower/OpenMP/OpenMP.cpp | 155 ++++++-----
.../scope-allocate-clause-unsupported.f90 | 76 ++++++
.../Lower/OpenMP/scope-allocate-clause.f90 | 122 +++++++++
flang/test/Lower/OpenMP/scope.f90 | 2 +-
flang/test/Lower/OpenMP/target-scope.f90 | 2 +-
mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp | 10 +-
.../OpenMP/OpenMPToLLVMIRTranslation.cpp | 127 ++++++---
mlir/test/Dialect/OpenMP/invalid.mlir | 90 +++++++
mlir/test/Dialect/OpenMP/ops.mlir | 15 ++
.../Target/LLVMIR/openmp-allocate-clause.mlir | 8 +-
.../LLVMIR/openmp-scope-allocate-clause.mlir | 241 ++++++++++++++++++
mlir/test/Target/LLVMIR/openmp-todo.mlir | 11 -
13 files changed, 737 insertions(+), 124 deletions(-)
create mode 100644 flang/test/Lower/OpenMP/Todo/scope-allocate-clause-unsupported.f90
create mode 100644 flang/test/Lower/OpenMP/scope-allocate-clause.f90
create mode 100644 mlir/test/Target/LLVMIR/openmp-scope-allocate-clause.mlir
diff --git a/flang/docs/OpenMPSupport.md b/flang/docs/OpenMPSupport.md
index 5a5285421e724..93fe57440c234 100644
--- a/flang/docs/OpenMPSupport.md
+++ b/flang/docs/OpenMPSupport.md
@@ -226,7 +226,7 @@ Parser/Semantics, MLIR, Lowering, or the OpenMPIRBuilder.
| otherwise clause on metadirectives | <span class="good">done</span> | | Lowering handles `OmpClause::Otherwise` as the fallback variant in `genMetadirective` (`flang/lib/Lower/OpenMP/OpenMP.cpp`); the legacy `default` clause spelling is also handled. | [llvm/llvm-project#194402](https://github.com/llvm/llvm-project/pull/194402), [llvm/llvm-project#194424](https://github.com/llvm/llvm-project/pull/194424) |
| doacross with omp_cur_iteration | <span class="progress">in progress</span> | dreachem | Implement parser/semantics validation for `omp_cur_iteration` placement and lowering of doacross dependence tokens, then add semantics+lowering tests for source/sink combinations. | |
| implicit map type for target enter and exit data | <span class="good">done</span> | | Target enter data applies implicit `to` mapping; target exit data applies implicit `from` mapping. Lowering via `getImplicitMapTypeAndKind` and MapInfoFinalization pass. Comprehensive test coverage in `target-enter-data-default-openmp52.f90` for OpenMP 5.2+ including allocatable descriptor handling. | [llvm/llvm-project#174665](https://github.com/llvm/llvm-project/pull/174665), [llvm/llvm-project#193851](https://github.com/llvm/llvm-project/pull/193851) |
-| allocate and firstprivate on scope directive | <span class="part">partial</span> | | Firstprivate on scope is fully supported and tested (`scope.f90`). Allocate clause on scope is parsed/lowered to MLIR but not yet implemented in LLVM IR translation layer (checkImplementationStatus in OpenMPToLLVMIRTranslation.cpp blocks it). | [llvm/llvm-project#193098](https://github.com/llvm/llvm-project/pull/193098) |
+| allocate and firstprivate on scope directive | <span class="part">partial</span> | | Firstprivate on scope is fully supported and tested (`scope.f90`). Host-side allocate clauses on scope use allocator-backed storage for fixed-size intrinsic scalar PRIVATE/FIRSTPRIVATE items with omitted/default/explicit allocator and ALIGN (`scope-allocate-clause.f90`, `openmp-scope-allocate-clause.mlir`). Arrays, descriptors, pointers, allocatables, derived types, common blocks, SAVE entities, and target/device scope allocation remain unsupported. | [llvm/llvm-project#193098](https://github.com/llvm/llvm-project/pull/193098) |
| loop consistency changes for order clause | <span class="none">unclaimed</span> | | Extend semantic loop-consistency checks for updated ORDER rules and add diagnostics tests for invalid nest/ordering combinations. | |
| keep original base pointer on map without matched candidate | <span class="none">unclaimed</span> | | Update map finalization so unmatched candidates preserve original base-pointer mapping semantics; add lowering tests for pointer-member mapping regressions. | |
| pure procedure support for certain directives | <span class="good">done</span> | ShashwathiNavada | Semantics enforce that metadirective, assumption directives (ASSUME/ASSUMES), NOTHING, ERROR, and the loop-transforming constructs TILE/UNROLL are permitted in a Fortran PURE procedure, in addition to the baseline of SIMD and declarative directives; other directives are rejected with a version-aware diagnostic. | [llvm/llvm-project#212676](https://github.com/llvm/llvm-project/pull/212676) |
diff --git a/flang/lib/Lower/OpenMP/OpenMP.cpp b/flang/lib/Lower/OpenMP/OpenMP.cpp
index becb05b5af2c2..c2dff13a00634 100644
--- a/flang/lib/Lower/OpenMP/OpenMP.cpp
+++ b/flang/lib/Lower/OpenMP/OpenMP.cpp
@@ -2607,7 +2607,7 @@ static void genScopeClauses(lower::AbstractConverter &converter,
mlir::omp::ScopeOperands &clauseOps,
llvm::SmallVectorImpl<Object> &reductionObjects) {
ClauseProcessor cp(converter, semaCtx, clauses);
- cp.processAllocate(clauseOps);
+ cp.processAllocate(clauseOps, /*supportAlignment=*/true);
cp.processNowait(clauseOps);
cp.processReduction(loc, clauseOps, reductionObjects);
}
@@ -3452,6 +3452,87 @@ genOrderedRegionOp(lower::AbstractConverter &converter, lower::SymMap &symTable,
queue, item, clauseOps);
}
+/// Private operand order follows privatization expansion rather than the
+/// ALLOCATE clause's list order.
+template <typename ClauseOpsT>
+static void
+mapAllocateClauseToPrivateSlots(mlir::Location loc, const List<Clause> &clauses,
+ const ObjectEntryBlockArgs &args,
+ ClauseOpsT &clauseOps,
+ bool requireAutomaticStorage = false) {
+ if (clauseOps.allocateVars.empty())
+ return;
+
+ llvm::DenseMap<const semantics::Symbol *, int64_t> privateSlots;
+ int64_t privateSlot = 0;
+ auto addPrivateSlot = [&](const semantics::Symbol &symbol) {
+ if (!privateSlots.try_emplace(&symbol.GetUltimate(), privateSlot).second)
+ fir::emitFatalError(
+ loc, "symbol with multiple private storage slots on one construct");
+ ++privateSlot;
+ };
+ for (const Object &object : args.priv.objects) {
+ const semantics::Symbol *symbol = object.sym();
+ if (!symbol)
+ fir::emitFatalError(loc, "private item without a semantic symbol");
+ // A privatized common block contributes one private operand per member,
+ // so slot numbering must follow the same expansion.
+ if (const auto *commonDetails =
+ symbol->detailsIf<semantics::CommonBlockDetails>()) {
+ for (const auto &member : commonDetails->objects())
+ addPrivateSlot(*member);
+ } else {
+ addPrivateSlot(*symbol);
+ }
+ }
+
+ llvm::DenseSet<const semantics::Symbol *> allocateSymbols;
+ for (const Clause &clause : clauses) {
+ if (clause.id != llvm::omp::Clause::OMPC_allocate)
+ continue;
+ const auto &allocate = std::get<clause::Allocate>(clause.u);
+ const auto &objects = std::get<ObjectList>(allocate.t);
+ for (const Object &object : objects) {
+ const semantics::Symbol *symbol = object.sym();
+ if (!symbol)
+ fir::emitFatalError(loc,
+ "ALLOCATE clause item without a semantic symbol");
+ const semantics::Symbol *ultimate = &symbol->GetUltimate();
+ if (!allocateSymbols.insert(ultimate).second)
+ TODO(loc, "ALLOCATE clause item appears more than once");
+
+ auto privateSlot = privateSlots.find(ultimate);
+ if (privateSlot == privateSlots.end())
+ fir::emitFatalError(
+ loc, "ALLOCATE clause item without private storage slot");
+
+ auto type = evaluate::DynamicType::From(*ultimate);
+ bool supportedDataSharing =
+ symbol->test(semantics::Symbol::Flag::OmpPrivate) ||
+ symbol->test(semantics::Symbol::Flag::OmpFirstPrivate);
+ bool supportedType = ultimate->Rank() == 0 &&
+ !semantics::IsAllocatableOrPointer(*ultimate) &&
+ type &&
+ type->category() != common::TypeCategory::Derived &&
+ !type->RequiresDescriptor() &&
+ !type->HasDeferredOrAssumedTypeParameter();
+ if (requireAutomaticStorage)
+ supportedType = supportedType && !semantics::IsSaved(*ultimate) &&
+ !semantics::FindCommonBlockContaining(*ultimate);
+ if (!supportedDataSharing || !supportedType)
+ TODO(loc,
+ "ALLOCATE clause currently supports only fixed-size intrinsic "
+ "scalar PRIVATE or FIRSTPRIVATE items");
+
+ clauseOps.allocatePrivateIndices.push_back(privateSlot->second);
+ }
+ }
+
+ if (clauseOps.allocatePrivateIndices.size() != clauseOps.allocateVars.size())
+ fir::emitFatalError(loc,
+ "incomplete ALLOCATE clause private storage mapping");
+}
+
static mlir::omp::ParallelOp
genParallelOp(lower::AbstractConverter &converter, lower::SymMap &symTable,
semantics::SemanticsContext &semaCtx,
@@ -3463,74 +3544,7 @@ genParallelOp(lower::AbstractConverter &converter, lower::SymMap &symTable,
assert((!enableDelayedPrivatization || dsp) &&
"expected valid DataSharingProcessor");
- if (!clauseOps.allocateVars.empty()) {
- llvm::DenseMap<const semantics::Symbol *, int64_t> privateSlots;
- int64_t privateSlot = 0;
- auto addPrivateSlot = [&](const semantics::Symbol &symbol) {
- if (!privateSlots.try_emplace(&symbol.GetUltimate(), privateSlot).second)
- fir::emitFatalError(
- loc, "symbol with multiple private storage slots on one construct");
- ++privateSlot;
- };
- for (const Object &object : args.priv.objects) {
- const semantics::Symbol *symbol = object.sym();
- if (!symbol)
- fir::emitFatalError(loc, "private item without a semantic symbol");
- // A privatized common block contributes one private operand per member,
- // so slot numbering must follow the same expansion.
- if (const auto *commonDetails =
- symbol->detailsIf<semantics::CommonBlockDetails>()) {
- for (const auto &member : commonDetails->objects())
- addPrivateSlot(*member);
- } else {
- addPrivateSlot(*symbol);
- }
- }
-
- llvm::DenseSet<const semantics::Symbol *> allocateSymbols;
- for (const Clause &clause : item->clauses) {
- if (clause.id != llvm::omp::Clause::OMPC_allocate)
- continue;
- const auto &allocate = std::get<clause::Allocate>(clause.u);
- const auto &objects = std::get<ObjectList>(allocate.t);
- for (const Object &object : objects) {
- const semantics::Symbol *symbol = object.sym();
- if (!symbol)
- fir::emitFatalError(loc,
- "ALLOCATE clause item without a semantic symbol");
- const semantics::Symbol *ultimate = &symbol->GetUltimate();
- if (!allocateSymbols.insert(ultimate).second)
- TODO(loc, "ALLOCATE clause item appears more than once");
-
- auto privateSlot = privateSlots.find(ultimate);
- if (privateSlot == privateSlots.end())
- fir::emitFatalError(
- loc, "ALLOCATE clause item without private storage slot");
-
- auto type = evaluate::DynamicType::From(*ultimate);
- bool supportedDataSharing =
- symbol->test(semantics::Symbol::Flag::OmpPrivate) ||
- symbol->test(semantics::Symbol::Flag::OmpFirstPrivate);
- bool supportedType =
- ultimate->Rank() == 0 &&
- !semantics::IsAllocatableOrPointer(*ultimate) && type &&
- type->category() != common::TypeCategory::Derived &&
- !type->RequiresDescriptor() &&
- !type->HasDeferredOrAssumedTypeParameter();
- if (!supportedDataSharing || !supportedType)
- TODO(loc,
- "ALLOCATE clause currently supports only fixed-size intrinsic "
- "scalar PRIVATE or FIRSTPRIVATE items");
-
- clauseOps.allocatePrivateIndices.push_back(privateSlot->second);
- }
- }
-
- if (clauseOps.allocatePrivateIndices.size() !=
- clauseOps.allocateVars.size())
- fir::emitFatalError(loc,
- "incomplete ALLOCATE clause private storage mapping");
- }
+ mapAllocateClauseToPrivateSlots(loc, item->clauses, args, clauseOps);
OpWithBodyGenInfo genInfo =
OpWithBodyGenInfo(converter, symTable, semaCtx, loc, eval,
@@ -3891,6 +3905,9 @@ genScopeOp(lower::AbstractConverter &converter, lower::SymMap &symTable,
args.reduction.objects = reductionObjects;
args.reduction.vars = clauseOps.reductionVars;
+ mapAllocateClauseToPrivateSlots(loc, item->clauses, args, clauseOps,
+ /*requireAutomaticStorage=*/true);
+
return genOpWithBody<mlir::omp::ScopeOp>(
OpWithBodyGenInfo(converter, symTable, semaCtx, loc, eval,
llvm::omp::Directive::OMPD_scope)
diff --git a/flang/test/Lower/OpenMP/Todo/scope-allocate-clause-unsupported.f90 b/flang/test/Lower/OpenMP/Todo/scope-allocate-clause-unsupported.f90
new file mode 100644
index 0000000000000..99e5fb46a1b9c
--- /dev/null
+++ b/flang/test/Lower/OpenMP/Todo/scope-allocate-clause-unsupported.f90
@@ -0,0 +1,76 @@
+! RUN: split-file %s %t
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/array.f90 2>&1 | FileCheck %s --check-prefix=ARRAY
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/derived.f90 2>&1 | FileCheck %s --check-prefix=DERIVED
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/pointer.f90 2>&1 | FileCheck %s --check-prefix=POINTER
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/allocatable.f90 2>&1 | FileCheck %s --check-prefix=ALLOCATABLE
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/assumed-length.f90 2>&1 | FileCheck %s --check-prefix=ASSUMED-LENGTH
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/save.f90 2>&1 | FileCheck %s --check-prefix=SAVE
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/common.f90 2>&1 | FileCheck %s --check-prefix=COMMON
+
+! ARRAY: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! DERIVED: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! POINTER: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! ALLOCATABLE: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! ASSUMED-LENGTH: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! SAVE: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! COMMON: The ALLOCATE clause requires that 'x' must be listed in a private data-sharing attribute clause on the same directive
+
+!--- array.f90
+subroutine array(x)
+ integer :: x(4)
+ !$omp scope private(x) allocate(align(64): x)
+ x = 1
+ !$omp end scope
+end subroutine
+
+!--- derived.f90
+subroutine derived()
+ type t
+ integer :: value
+ end type
+ type(t) :: x
+ !$omp scope private(x) allocate(x)
+ x%value = 1
+ !$omp end scope
+end subroutine
+
+!--- pointer.f90
+subroutine pointer(x)
+ integer, pointer :: x
+ !$omp scope private(x) allocate(x)
+ x = 1
+ !$omp end scope
+end subroutine
+
+!--- allocatable.f90
+subroutine allocatable(x)
+ integer, allocatable :: x
+ !$omp scope private(x) allocate(x)
+ x = 1
+ !$omp end scope
+end subroutine
+
+!--- assumed-length.f90
+subroutine assumed_length(x)
+ character(*) :: x
+ !$omp scope private(x) allocate(x)
+ x = "test"
+ !$omp end scope
+end subroutine
+
+!--- save.f90
+subroutine save_entity()
+ integer, save :: x
+ !$omp scope private(x) allocate(x)
+ x = 1
+ !$omp end scope
+end subroutine
+
+!--- common.f90
+subroutine common_block()
+ integer :: x
+ common /block/ x
+ !$omp scope private(/block/) allocate(x)
+ x = 1
+ !$omp end scope
+end subroutine
diff --git a/flang/test/Lower/OpenMP/scope-allocate-clause.f90 b/flang/test/Lower/OpenMP/scope-allocate-clause.f90
new file mode 100644
index 0000000000000..a9348865cb8d8
--- /dev/null
+++ b/flang/test/Lower/OpenMP/scope-allocate-clause.f90
@@ -0,0 +1,122 @@
+! This test checks lowering of the ALLOCATE clause on the OpenMP SCOPE
+! construct: PRIVATE with omitted/default allocator, PRIVATE with an explicit
+! allocator, FIRSTPRIVATE with allocator-backed storage, ALIGN, and an
+! allocation-list order that differs from the private-list order.
+
+! RUN: %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %s 2>&1 | FileCheck %s
+! RUN: %flang_fc1 -emit-llvm %openmp_flags -fopenmp-version=52 -o - %s 2>&1 | FileCheck %s --check-prefix=LLVM
+! RUN: not %flang_fc1 -fsyntax-only %openmp_flags -fopenmp-version=51 %s 2>&1 | FileCheck %s --check-prefix=VERSION51
+
+! VERSION51: error: {{.*}}ALLOCATE
+
+subroutine scope_allocator_omitted(x, y)
+ integer :: x, y
+ !$omp scope private(x, y) allocate(x)
+ x = 1
+ y = 2
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_omitted
+! CHECK: %[[NULL_ALLOC:.*]] = arith.constant 0 : i32
+! CHECK: omp.scope allocate(%[[NULL_ALLOC]] : i32 -> %[[X:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[X]]#0 -> %[[X_PRIVATE:.*]], {{.*}} -> %[[Y_PRIVATE:.*]] : !fir.ref<i32>, !fir.ref<i32>) {
+
+subroutine scope_allocator_explicit(x, allocator)
+ use iso_c_binding, only : c_intptr_t
+ integer :: x
+ integer(c_intptr_t), intent(in) :: allocator
+ !$omp scope private(x) allocate(allocator(allocator): x)
+ x = 1
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_explicit
+! CHECK: %[[ALLOCATOR:.*]] = fir.load %{{.*}} : !fir.ref<i64>
+! CHECK: omp.scope allocate(%[[ALLOCATOR]] : i64 -> %[[X:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[X]]#0 -> {{.*}} : !fir.ref<i32>) {
+
+subroutine scope_allocator_firstprivate(x)
+ integer :: x
+ !$omp scope firstprivate(x) allocate(x)
+ x = x + 1
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_firstprivate
+! CHECK: omp.scope allocate({{.*}} : i32 -> %[[X:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[X]]#0 -> %[[X_PRIVATE:.*]] : !fir.ref<i32>) {
+! CHECK: %[[X_DECL:.*]]:2 = hlfir.declare %[[X_PRIVATE]]
+! CHECK: hlfir.assign %{{.*}} to %[[X_DECL]]#0
+
+subroutine scope_allocator_align(x, y)
+ integer :: x, y
+ !$omp scope private(x, y) allocate(x) allocate(align(64): y)
+ x = 1
+ y = 2
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_align
+! CHECK: omp.scope allocate(
+! CHECK-SAME: allocate_alignments([0, 64]) allocate_private_indices([0, 1])
+! CHECK-SAME: private(
+
+subroutine scope_allocator_order(x, y, z, allocator_xy, allocator_z)
+ use iso_c_binding, only : c_intptr_t
+ integer :: x, y, z
+ integer(c_intptr_t), intent(in) :: allocator_xy, allocator_z
+ !$omp scope private(x, y, z) &
+ !$omp& allocate(allocator(allocator_xy): y, x) &
+ !$omp& allocate(allocator(allocator_z): z)
+ x = 1
+ y = 2
+ z = 3
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_order
+! CHECK: omp.scope allocate(
+! CHECK-SAME: allocate_private_indices([1, 0, 2])
+! CHECK-SAME: private(
+
+subroutine scope_allocator_dynamic(x, allocators, n)
+ use iso_c_binding, only : c_intptr_t
+ integer, intent(in) :: n
+ integer(c_intptr_t), intent(in) :: allocators(n)
+ integer :: x, i
+ do i = 1, n
+ !$omp scope firstprivate(x) allocate(allocator(allocators(i)): x)
+ x = x + i
+ !$omp end scope
+ end do
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_dynamic
+! CHECK: fir.do_loop
+! CHECK: %[[DYNAMIC_ALLOCATOR:.*]] = fir.load %{{.*}} : !fir.ref<i64>
+! CHECK: omp.scope allocate(%[[DYNAMIC_ALLOCATOR]] : i64 -> %[[X:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[X]]#0 -> {{.*}} : !fir.ref<i32>) {
+
+! LLVM-LABEL: define void @scope_allocator_dynamic_
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM: omp.region.after_alloca:
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM: br label %[[LOOP:[0-9]+]]
+! LLVM: [[LOOP]]:
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM: br i1 {{.*}}, label %[[BODY:[0-9]+]], label %[[EXIT:[0-9]+]]
+! LLVM: [[BODY]]:
+! LLVM: %[[DYNAMIC_ALLOCATOR:.*]] = load i64
+! LLVM: %[[ALLOCATOR_HANDLE:.*]] = inttoptr i64 %[[DYNAMIC_ALLOCATOR]] to ptr
+! LLVM: %[[ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr %[[ALLOCATOR_HANDLE]])
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM-NOT: call void @__kmpc_free
+! LLVM: store i32 {{.*}}, ptr %[[ALLOC]]
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM-NOT: call void @__kmpc_free
+! LLVM: call void @__kmpc_free({{.*}}, ptr %[[ALLOC]], ptr %[[ALLOCATOR_HANDLE]])
+! LLVM-NOT: call void @__kmpc_free
+! LLVM: br label %[[LOOP]]
+! LLVM: [[EXIT]]:
+! LLVM: ret void
diff --git a/flang/test/Lower/OpenMP/scope.f90 b/flang/test/Lower/OpenMP/scope.f90
index baa5ed493052b..2593bf5b6253c 100644
--- a/flang/test/Lower/OpenMP/scope.f90
+++ b/flang/test/Lower/OpenMP/scope.f90
@@ -103,7 +103,7 @@ subroutine omp_scope_allocate()
! CHECK: hlfir.declare %{{.*}} {uniq_name = "_QFomp_scope_allocateEi"}
i = 0
- ! CHECK: omp.scope allocate(%{{.*}} : i32 -> %{{.*}}#0 : !fir.ref<i32>) private(@_QFomp_scope_allocateEi_private_i32 %{{.*}}#0 -> %[[APRIV:.*]] : !fir.ref<i32>) {
+ ! CHECK: omp.scope allocate(%{{.*}} : i32 -> %{{.*}}#0 : !fir.ref<i32>) allocate_private_indices([0]) private(@_QFomp_scope_allocateEi_private_i32 %{{.*}}#0 -> %[[APRIV:.*]] : !fir.ref<i32>) {
! CHECK: %[[ADECL:.*]]:2 = hlfir.declare %[[APRIV]] {uniq_name = "_QFomp_scope_allocateEi"}
!$omp scope private(i) allocate(i)
! CHECK: hlfir.assign %{{.*}} to %[[ADECL]]#0 : i32, !fir.ref<i32>
diff --git a/flang/test/Lower/OpenMP/target-scope.f90 b/flang/test/Lower/OpenMP/target-scope.f90
index de3bee3bdcacb..0684c529c96b9 100644
--- a/flang/test/Lower/OpenMP/target-scope.f90
+++ b/flang/test/Lower/OpenMP/target-scope.f90
@@ -123,7 +123,7 @@ subroutine target_scope_allocate()
!$omp target
! CHECK: omp.target kernel_type(generic) map_entries(%{{.*}} -> %[[IARG:.*]] : !fir.ref<i32>) {
! CHECK: hlfir.declare %[[IARG]] {uniq_name = "_QFtarget_scope_allocateEi"}
- ! CHECK: omp.scope allocate(%{{.*}} : i32 -> %{{.*}}#0 : !fir.ref<i32>) private(@_QFtarget_scope_allocateEi_private_i32 %{{.*}}#0 -> %[[APRIV:.*]] : !fir.ref<i32>) {
+ ! CHECK: omp.scope allocate(%{{.*}} : i32 -> %{{.*}}#0 : !fir.ref<i32>) allocate_private_indices([0]) private(@_QFtarget_scope_allocateEi_private_i32 %{{.*}}#0 -> %[[APRIV:.*]] : !fir.ref<i32>) {
! CHECK: %[[ADECL:.*]]:2 = hlfir.declare %[[APRIV]] {uniq_name = "_QFtarget_scope_allocateEi"}
!$omp scope private(i) allocate(i)
! CHECK: hlfir.assign %{{.*}} to %[[ADECL]]#0 : i32, !fir.ref<i32>
diff --git a/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp b/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
index 1f210ef60fe39..4a38acf5d398d 100644
--- a/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
+++ b/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
@@ -707,9 +707,12 @@ static LogicalResult verifyAllocateClause(
return op->emitError(
"expected as many allocate private indices as allocate variables");
+ DenseSet<Value> usedAllocateVars;
DenseSet<int64_t> usedPrivateSlots;
for (auto [allocateVar, privateIndex] :
llvm::zip_equal(allocateVars, indices)) {
+ if (!usedAllocateVars.insert(allocateVar).second)
+ return op->emitError("allocate variable appears more than once");
if (privateIndex < 0 ||
static_cast<uint64_t>(privateIndex) >= privateVars.size())
return op->emitError("allocate private index is out of range");
@@ -723,6 +726,10 @@ static LogicalResult verifyAllocateClause(
<< "type mismatch between allocate variable and private variable "
"at index "
<< privateIndex;
+ if (allocateVar != privateVar)
+ return op->emitError()
+ << "allocate variable does not match private variable at index "
+ << privateIndex;
if (!privateSyms ||
static_cast<uint64_t>(privateIndex) >= privateSyms.size())
@@ -3276,7 +3283,8 @@ LogicalResult ScopeOp::verify() {
if (failed(verifyAllocateClause(
getOperation(), getAllocateVars(), getAllocatorVars(),
getAllocateAlignmentsAttr(), getAllocatePrivateIndicesAttr(),
- getPrivateVars(), getPrivateSymsAttr())))
+ getPrivateVars(), getPrivateSymsAttr(),
+ /*requirePrivateIndices=*/true)))
return failure();
if (failed(verifyPrivateVarList(*this)))
diff --git a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
index fd5991a5b177e..5957a10040ac3 100644
--- a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
@@ -549,7 +549,6 @@ static LogicalResult checkImplementationStatus(Operation &op) {
checkReduction(op, result);
})
.Case([&](omp::ScopeOp op) {
- checkAllocate(op, result);
checkReduction(op, result);
})
.Case([&](omp::SingleOp op) {
@@ -1933,16 +1932,44 @@ initPrivateVars(llvm::IRBuilderBase &builder,
return llvm::Error::success();
}
+static LogicalResult
+convertAllocatorVars(Operation &op, ValueRange allocatorVars,
+ llvm::IRBuilderBase &builder,
+ LLVM::ModuleTranslation &moduleTranslation,
+ PrivateVarsInfo &privateVarsInfo) {
+ for (Value allocatorVar : allocatorVars) {
+ if (privateVarsInfo.convertedAllocators.contains(allocatorVar))
+ continue;
+
+ llvm::Value *allocator = moduleTranslation.lookupValue(allocatorVar);
+ if (!allocator)
+ return op.emitError("failed to translate OpenMP allocator operand");
+ if (allocator->getType()->isIntegerTy())
+ allocator = builder.CreateIntToPtr(allocator, builder.getPtrTy());
+ else if (allocator->getType()->isPointerTy())
+ allocator = builder.CreatePointerBitCastOrAddrSpaceCast(
+ allocator, builder.getPtrTy());
+ else
+ return op.emitError(
+ "OpenMP allocator operand must have integer or pointer type");
+
+ privateVarsInfo.convertedAllocators.try_emplace(allocatorVar, allocator);
+ }
+ return success();
+}
+
/// Allocate and initialize delayed private variables. Returns the basic block
/// which comes after all of these allocations. llvm::Value * for each of these
/// private variables are populated in llvmPrivateVars.
template <typename T>
-static llvm::Expected<llvm::BasicBlock *>
-allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
- LLVM::ModuleTranslation &moduleTranslation,
- PrivateVarsInfo &privateVarsInfo,
- const llvm::OpenMPIRBuilder::InsertPointTy &allocaIP,
- llvm::DenseMap<Value, Value> *mappedPrivateVars = nullptr) {
+static llvm::Expected<llvm::BasicBlock *> allocatePrivateVars(
+ T op, llvm::IRBuilderBase &builder,
+ LLVM::ModuleTranslation &moduleTranslation,
+ PrivateVarsInfo &privateVarsInfo,
+ const llvm::OpenMPIRBuilder::InsertPointTy &allocaIP,
+ llvm::DenseMap<Value, Value> *mappedPrivateVars = nullptr,
+ std::optional<llvm::OpenMPIRBuilder::InsertPointTy> allocatorIP =
+ std::nullopt) {
// Allocate private vars
llvm::Instruction *allocaTerminator = allocaIP.getBlock()->getTerminator();
splitBB(llvm::OpenMPIRBuilder::InsertPointTy(allocaIP.getBlock(),
@@ -1950,13 +1977,31 @@ allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
true, allocaTerminator->getStableDebugLoc(),
"omp.region.after_alloca");
- llvm::IRBuilderBase::InsertPointGuard guard(builder);
+ llvm::Instruction *allocatorTerminator = nullptr;
+ llvm::BasicBlock *afterAllocatorAllocations = nullptr;
+ if (allocatorIP) {
+ allocatorTerminator = allocatorIP->getBlock()->getTerminator();
+ afterAllocatorAllocations = splitBB(
+ llvm::OpenMPIRBuilder::InsertPointTy(
+ allocatorIP->getBlock(), allocatorTerminator->getIterator()),
+ true, allocatorTerminator->getStableDebugLoc(),
+ "omp.region.after_allocate");
+ }
+
+ std::optional<llvm::IRBuilderBase::InsertPointGuard> guard;
+ if (!allocatorIP)
+ guard.emplace(builder);
// Update the allocaTerminator since the alloca block was split above.
allocaTerminator = allocaIP.getBlock()->getTerminator();
builder.SetInsertPoint(allocaTerminator);
// The new terminator is an uncondition branch created by the splitBB above.
assert(allocaTerminator->getNumSuccessors() == 1 &&
"This is an unconditional branch created by splitBB");
+ if (allocatorIP) {
+ allocatorTerminator = allocatorIP->getBlock()->getTerminator();
+ assert(allocatorTerminator->getNumSuccessors() == 1 &&
+ "This is an unconditional branch created by splitBB");
+ }
llvm::DataLayout dataLayout = builder.GetInsertBlock()->getDataLayout();
llvm::BasicBlock *afterAllocas = allocaTerminator->getSuccessor(0);
@@ -1973,7 +2018,8 @@ allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
-1);
ValueRange allocatorVars;
DenseI64ArrayAttr allocateAlignments;
- if constexpr (std::is_same_v<T, omp::ParallelOp>) {
+ if constexpr (std::is_same_v<T, omp::ParallelOp> ||
+ std::is_same_v<T, omp::ScopeOp>) {
allocatorVars = op.getAllocatorVars();
allocateAlignments = op.getAllocateAlignmentsAttr();
if (auto privateIndices = op.getAllocatePrivateIndicesAttr())
@@ -1988,14 +2034,16 @@ allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
auto [privDecl, mlirPrivVar, blockArg] = tuple;
llvm::Type *llvmAllocType =
moduleTranslation.convertType(privDecl.getType());
- builder.SetInsertPoint(allocaIP.getBlock()->getTerminator());
llvm::Value *llvmPrivateVar = nullptr;
int64_t allocateIndex = allocateItemForPrivate[privateIndex];
+ builder.SetInsertPoint(allocateIndex >= 0 && allocatorTerminator
+ ? allocatorTerminator
+ : allocaTerminator);
if (allocateIndex >= 0) {
if (mightUseDeviceSharedMem ||
op->template getParentOfType<omp::TargetOp>())
return llvm::createStringError(
- "allocate clause on a device parallel region is not supported");
+ "allocate clause in an OpenMP device context is not supported");
if (!llvmAllocType->isSized())
return llvm::createStringError(
"allocate clause private type must have a fixed size");
@@ -2058,7 +2106,7 @@ allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
privateVarsInfo.llvmVars.push_back(llvmPrivateVar);
}
- return afterAllocas;
+ return afterAllocatorAllocations ? afterAllocatorAllocations : afterAllocas;
}
/// This can't always be determined statically, but when we can, it is good to
@@ -2382,10 +2430,14 @@ convertOmpScope(omp::ScopeOp &scopeOp, llvm::IRBuilderBase &builder,
assert(isByRef.size() == scopeOp.getNumReductionVars());
PrivateVarsInfo privateVarsInfo(scopeOp);
+ if (failed(convertAllocatorVars(*scopeOp, scopeOp.getAllocatorVars(), builder,
+ moduleTranslation, privateVarsInfo)))
+ return failure();
SmallVector<omp::DeclareReductionOp> reductionDecls;
collectReductionDecls(scopeOp, reductionDecls);
- InsertPointTy allocaIP = findAllocInsertPoints(builder, moduleTranslation);
+ InsertPointTy privateAllocaIP =
+ findAllocInsertPoints(builder, moduleTranslation);
SmallVector<llvm::Value *> privateReductionVariables(
scopeOp.getNumReductionVars());
@@ -2394,14 +2446,15 @@ convertOmpScope(omp::ScopeOp &scopeOp, llvm::IRBuilderBase &builder,
MutableArrayRef<BlockArgument> reductionArgs =
cast<omp::BlockArgOpenMPOpInterface>(*scopeOp).getReductionBlockArgs();
- // Allocate private vars before the scope body
- llvm::Expected<llvm::BasicBlock *> afterAllocas = allocatePrivateVars(
- scopeOp, builder, moduleTranslation, privateVarsInfo, allocaIP);
- if (failed(handleError(afterAllocas, *scopeOp)))
- return failure();
+ if (scopeOp.getAllocateVars().empty()) {
+ llvm::Expected<llvm::BasicBlock *> afterAllocas = allocatePrivateVars(
+ scopeOp, builder, moduleTranslation, privateVarsInfo, privateAllocaIP);
+ if (failed(handleError(afterAllocas, *scopeOp)))
+ return failure();
+ }
if (failed(allocAndInitializeReductionVars(
- scopeOp, reductionArgs, builder, moduleTranslation, allocaIP,
+ scopeOp, reductionArgs, builder, moduleTranslation, privateAllocaIP,
reductionDecls, privateReductionVariables, reductionVariableMap,
isByRef)))
return failure();
@@ -2409,7 +2462,18 @@ convertOmpScope(omp::ScopeOp &scopeOp, llvm::IRBuilderBase &builder,
auto bodyCB =
[&](InsertPointTy allocaIP, InsertPointTy codeGenIP,
llvm::ArrayRef<llvm::BasicBlock *> deallocBlocks) -> llvm::Error {
- builder.restoreIP(codeGenIP);
+ if (!scopeOp.getAllocateVars().empty()) {
+ // Runtime storage must be allocated on each dynamic Scope entry. Ordinary
+ // private allocas still use the enclosing alloca insertion point.
+ llvm::Expected<llvm::BasicBlock *> afterAllocas = allocatePrivateVars(
+ scopeOp, builder, moduleTranslation, privateVarsInfo, privateAllocaIP,
+ /*mappedPrivateVars=*/nullptr, codeGenIP);
+ if (handleError(afterAllocas, *scopeOp).failed())
+ return llvm::make_error<PreviouslyReportedError>();
+ builder.SetInsertPoint(afterAllocas.get()->getTerminator());
+ } else {
+ builder.restoreIP(codeGenIP);
+ }
if (handleError(
initPrivateVars(builder, moduleTranslation, privateVarsInfo),
@@ -2449,7 +2513,7 @@ convertOmpScope(omp::ScopeOp &scopeOp, llvm::IRBuilderBase &builder,
// Process the reductions if required.
return createReductionsAndCleanup(
- scopeOp, builder, moduleTranslation, allocaIP, reductionDecls,
+ scopeOp, builder, moduleTranslation, privateAllocaIP, reductionDecls,
privateReductionVariables, isByRef, scopeOp.getNowait(),
/*isTeamsReduction=*/false);
}
@@ -4864,24 +4928,9 @@ convertOmpParallel(omp::ParallelOp opInst, llvm::IRBuilderBase &builder,
return failure();
PrivateVarsInfo privateVarsInfo(opInst);
- for (Value allocatorVar : opInst.getAllocatorVars()) {
- if (privateVarsInfo.convertedAllocators.contains(allocatorVar))
- continue;
-
- llvm::Value *allocator = moduleTranslation.lookupValue(allocatorVar);
- if (!allocator)
- return opInst.emitError("failed to translate OpenMP allocator operand");
- if (allocator->getType()->isIntegerTy())
- allocator = builder.CreateIntToPtr(allocator, builder.getPtrTy());
- else if (allocator->getType()->isPointerTy())
- allocator = builder.CreatePointerBitCastOrAddrSpaceCast(
- allocator, builder.getPtrTy());
- else
- return opInst.emitError(
- "OpenMP allocator operand must have integer or pointer type");
-
- privateVarsInfo.convertedAllocators.try_emplace(allocatorVar, allocator);
- }
+ if (failed(convertAllocatorVars(*opInst, opInst.getAllocatorVars(), builder,
+ moduleTranslation, privateVarsInfo)))
+ return failure();
// Collect reduction declarations
SmallVector<omp::DeclareReductionOp> reductionDecls;
diff --git a/mlir/test/Dialect/OpenMP/invalid.mlir b/mlir/test/Dialect/OpenMP/invalid.mlir
index 5d47b78dce48e..0a677c54cd61d 100644
--- a/mlir/test/Dialect/OpenMP/invalid.mlir
+++ b/mlir/test/Dialect/OpenMP/invalid.mlir
@@ -3530,6 +3530,96 @@ func.func @omp_parallel_allocate_type_mismatch(
// -----
+omp.private {type = private} @scope_allocate_private : i32
+
+func.func @omp_scope_allocate_missing_map(%allocator : i64, %var : !llvm.ptr) {
+ // expected-error @below {{expected an allocate private index for each allocate variable}}
+ omp.scope allocate(%allocator : i64 -> %var : !llvm.ptr)
+ private(@scope_allocate_private %var -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
+omp.private {type = private} @scope_allocate_private : i32
+
+func.func @omp_scope_allocate_map_range(%allocator : i64, %var : !llvm.ptr) {
+ // expected-error @below {{allocate private index is out of range}}
+ omp.scope allocate(%allocator : i64 -> %var : !llvm.ptr) allocate_private_indices([1])
+ private(@scope_allocate_private %var -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
+omp.private {type = private} @scope_x_private : i32
+omp.private {type = private} @scope_y_private : i32
+
+func.func @omp_scope_allocate_map_duplicate(
+ %allocator : i64, %x : !llvm.ptr, %y : !llvm.ptr) {
+ // expected-error @below {{allocate private index refers to a private variable more than once}}
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr,
+ %allocator : i64 -> %y : !llvm.ptr) allocate_private_indices([0, 0])
+ private(@scope_x_private %x -> %x_private,
+ @scope_y_private %y -> %y_private : !llvm.ptr, !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
+omp.private {type = private} @scope_allocate_private : i32
+
+func.func @omp_scope_allocate_type_mismatch(
+ %allocator : i64, %allocate_var : i64, %private_var : !llvm.ptr) {
+ // expected-error @below {{type mismatch between allocate variable and private variable at index 0}}
+ omp.scope allocate(%allocator : i64 -> %allocate_var : i64) allocate_private_indices([0])
+ private(@scope_allocate_private %private_var -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
+omp.private {type = private} @scope_x_private : i32
+omp.private {type = private} @scope_y_private : i32
+
+func.func @omp_scope_allocate_wrong_private_slot(
+ %allocator : i64, %x : !llvm.ptr, %y : !llvm.ptr) {
+ // expected-error @below {{allocate variable does not match private variable at index 1}}
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([1])
+ private(@scope_x_private %x -> %x_private,
+ @scope_y_private %y -> %y_private : !llvm.ptr, !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
+omp.private {type = private} @scope_x_private : i32
+omp.private {type = private} @scope_y_private : i32
+
+func.func @omp_scope_allocate_duplicate_operand(
+ %allocator : i64, %x : !llvm.ptr, %y : !llvm.ptr) {
+ // expected-error @below {{allocate variable appears more than once}}
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr,
+ %allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([0, 1])
+ private(@scope_x_private %x -> %x_private,
+ @scope_y_private %y -> %y_private : !llvm.ptr, !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
func.func @omp_distribute_nested_wrapper(%lb: index, %ub: index, %step: index) -> () {
// expected-error @below {{an 'omp.wsloop' nested wrapper is only allowed when a composite 'omp.parallel' is the direct parent}}
omp.distribute {
diff --git a/mlir/test/Dialect/OpenMP/ops.mlir b/mlir/test/Dialect/OpenMP/ops.mlir
index a0aff2cdfda0e..632b1f05d86cc 100644
--- a/mlir/test/Dialect/OpenMP/ops.mlir
+++ b/mlir/test/Dialect/OpenMP/ops.mlir
@@ -218,6 +218,21 @@ func.func @omp_parallel_pretty(%data_var : memref<i32>, %if_cond : i1, %num_thre
return
}
+omp.private {type = private} @scope_allocate_private : memref<i32>
+
+// CHECK-LABEL: omp_scope_pretty
+func.func @omp_scope_pretty(%data_var : memref<i32>, %allocator : si32) -> () {
+ // CHECK: omp.scope allocate(
+ // CHECK-SAME: allocate_alignments([64]) allocate_private_indices([0])
+ // CHECK-SAME: private(
+ omp.scope allocate(%allocator : si32 -> %data_var : memref<i32>) allocate_alignments([64]) allocate_private_indices([0])
+ private(@scope_allocate_private %data_var -> %private : memref<i32>) {
+ omp.terminator
+ }
+
+ return
+}
+
// CHECK-LABEL: omp_loop_nest
func.func @omp_loop_nest(%lb : index, %ub : index, %step : index) -> () {
omp.wsloop {
diff --git a/mlir/test/Target/LLVMIR/openmp-allocate-clause.mlir b/mlir/test/Target/LLVMIR/openmp-allocate-clause.mlir
index 8ae2ce9f4a183..6ea3e7c077b8c 100644
--- a/mlir/test/Target/LLVMIR/openmp-allocate-clause.mlir
+++ b/mlir/test/Target/LLVMIR/openmp-allocate-clause.mlir
@@ -97,12 +97,18 @@ llvm.func @allocator_reverse_free(%x: !llvm.ptr, %y: !llvm.ptr,
// CHECK-LABEL: define internal void @allocator_reverse_free..omp_par
// CHECK: %[[CAPTURED_X:.*]] = load ptr, ptr %{{.*}}, align 8
// CHECK: %[[CAPTURED_Y:.*]] = load ptr, ptr %{{.*}}, align 8
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
// CHECK: %[[X_ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 64, i64 4, ptr %[[CAPTURED_X]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
// CHECK: %[[Y_ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 128, i64 4, ptr %[[CAPTURED_Y]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
// CHECK: store i32 1, ptr %[[X_ALLOC]], align 4
// CHECK: store i32 1, ptr %[[Y_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
// CHECK: call void @private_dealloc(ptr %[[X_ALLOC]])
+// CHECK-NOT: call void @__kmpc_free
// CHECK: call void @__kmpc_free({{.*}}, ptr %[[Y_ALLOC]], ptr %[[CAPTURED_Y]])
+// CHECK-NOT: call void @__kmpc_free
// CHECK: call void @__kmpc_free({{.*}}, ptr %[[X_ALLOC]], ptr %[[CAPTURED_X]])
// CHECK-NOT: call void @__kmpc_free
@@ -304,5 +310,5 @@ llvm.func @allocator_device() {
llvm.return
}
-// DEVICE: allocate clause on a device parallel region is not supported
+// DEVICE: allocate clause in an OpenMP device context is not supported
// DEVICE: LLVM Translation failed for operation: omp.parallel
diff --git a/mlir/test/Target/LLVMIR/openmp-scope-allocate-clause.mlir b/mlir/test/Target/LLVMIR/openmp-scope-allocate-clause.mlir
new file mode 100644
index 0000000000000..adb295149c529
--- /dev/null
+++ b/mlir/test/Target/LLVMIR/openmp-scope-allocate-clause.mlir
@@ -0,0 +1,241 @@
+// RUN: split-file %s %t
+// RUN: mlir-translate -mlir-to-llvmir -split-input-file %t/valid.mlir | FileCheck %s
+// RUN: not mlir-translate -mlir-to-llvmir %t/device.mlir 2>&1 | FileCheck %s --check-prefix=DEVICE
+
+//--- valid.mlir
+
+omp.private {type = private} @x.private : i32
+omp.private {type = private} @y.private : i32
+
+llvm.func @scope_allocator_unaligned(%x: !llvm.ptr, %y: !llvm.ptr) {
+ %null = llvm.mlir.constant(0 : i64) : i64
+ omp.scope allocate(%null : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@x.private %x -> %x.private,
+ @y.private %y -> %y.private : !llvm.ptr, !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ llvm.store %one, %y.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_unaligned
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[Y_ALLOCA:.*]] = alloca i32, align 4
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: %[[UNALIGNED:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: store i32 1, ptr %[[UNALIGNED]], align 4
+// CHECK: store i32 1, ptr %[[Y_ALLOCA]], align 4
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[UNALIGNED]], ptr null)
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: ret void
+
+// -----
+
+omp.private {type = private} @constant.private : i32
+
+llvm.func @scope_allocator_constant(%x: !llvm.ptr) {
+ %allocator = llvm.mlir.constant(3 : i64) : i64
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@constant.private %x -> %x.private : !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_constant
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[CONSTANT_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr inttoptr (i64 3 to ptr))
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 1, ptr %[[CONSTANT_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[CONSTANT_ALLOC]], ptr inttoptr (i64 3 to ptr))
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: ret void
+
+// -----
+
+llvm.func @private_dealloc(!llvm.ptr)
+
+omp.private {type = firstprivate} @x.private : i32 copy {
+^bb0(%original: !llvm.ptr, %private: !llvm.ptr):
+ %value = llvm.load %original : !llvm.ptr -> i32
+ llvm.store %value, %private : i32, !llvm.ptr
+ omp.yield(%private : !llvm.ptr)
+} dealloc {
+^bb0(%private: !llvm.ptr):
+ llvm.call @private_dealloc(%private) : (!llvm.ptr) -> ()
+ omp.yield
+}
+
+llvm.func @scope_allocator_firstprivate_aligned(%x: !llvm.ptr, %allocator: i64) {
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_alignments([64]) allocate_private_indices([0])
+ private(@x.private %x -> %x.private : !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_firstprivate_aligned
+// CHECK: %[[ALLOCATOR:.*]] = inttoptr i64 %{{.*}} to ptr
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 64, i64 4, ptr %[[ALLOCATOR]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[ORIGINAL:.*]] = load i32, ptr %{{.*}}, align 4
+// CHECK: store i32 %[[ORIGINAL]], ptr %[[ALLOC]], align 4
+// CHECK: store i32 1, ptr %[[ALLOC]], align 4
+// CHECK: call void @private_dealloc(ptr %[[ALLOC]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[ALLOC]], ptr %[[ALLOCATOR]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: ret void
+
+// -----
+
+omp.private {type = private} @x.private : i32
+omp.private {type = private} @y.private : i32
+
+llvm.func @scope_allocator_order(%x: !llvm.ptr, %y: !llvm.ptr,
+ %allocator.x: i64, %allocator.y: i64) {
+ omp.scope allocate(%allocator.y : i64 -> %y : !llvm.ptr,
+ %allocator.x : i64 -> %x : !llvm.ptr) allocate_alignments([128, 64]) allocate_private_indices([1, 0])
+ private(@x.private %x -> %x.private,
+ @y.private %y -> %y.private : !llvm.ptr, !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ llvm.store %one, %y.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// Allocation-list order (y, x) differs from private-list order (x, y); the
+// allocate_private_indices mapping must still route each allocation to the
+// correct private slot, and cleanup must free exactly once per allocation.
+// CHECK-LABEL: define void @scope_allocator_order
+// CHECK: %[[ALLOCATOR_Y:.*]] = inttoptr i64 %{{.*}} to ptr
+// CHECK: %[[ALLOCATOR_X:.*]] = inttoptr i64 %{{.*}} to ptr
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[X_ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 64, i64 4, ptr %[[ALLOCATOR_X]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[Y_ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 128, i64 4, ptr %[[ALLOCATOR_Y]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 1, ptr %[[X_ALLOC]], align 4
+// CHECK: store i32 1, ptr %[[Y_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[Y_ALLOC]], ptr %[[ALLOCATOR_Y]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[X_ALLOC]], ptr %[[ALLOCATOR_X]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: ret void
+
+// -----
+
+omp.private {type = private} @conditional.private : i32
+
+llvm.func @scope_allocator_conditional(%x: !llvm.ptr, %allocator: i64,
+ %condition: i1) {
+ llvm.cond_br %condition, ^scope, ^exit
+^scope:
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@conditional.private %x -> %x.private : !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.br ^exit
+^exit:
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_conditional
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: br i1 {{.*}}, label %[[SCOPE:.*]], label %[[EXIT:.*]]
+// CHECK: [[SCOPE]]:
+// CHECK: %[[CONDITIONAL_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr %{{.*}})
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 1, ptr %[[CONDITIONAL_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[CONDITIONAL_ALLOC]], ptr %{{.*}})
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: br label %[[EXIT]]
+// CHECK: [[EXIT]]:
+// CHECK: ret void
+
+// -----
+
+omp.private {type = private} @loop.private : i32
+
+llvm.func @scope_allocator_loop(%x: !llvm.ptr, %allocator.base: i64, %n: i64) {
+ %zero = llvm.mlir.constant(0 : i64) : i64
+ %one = llvm.mlir.constant(1 : i64) : i64
+ llvm.br ^loop(%zero : i64)
+^loop(%i: i64):
+ %condition = llvm.icmp "slt" %i, %n : i64
+ llvm.cond_br %condition, ^body, ^exit
+^body:
+ %allocator = llvm.add %allocator.base, %i : i64
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@loop.private %x -> %x.private : !llvm.ptr) {
+ %value = llvm.trunc %i : i64 to i32
+ llvm.store %value, %x.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ %next = llvm.add %i, %one : i64
+ llvm.br ^loop(%next : i64)
+^exit:
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_loop
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: omp.region.after_alloca:
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: br label %[[LOOP:[0-9]+]]
+// CHECK: [[LOOP]]:
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: br i1 {{.*}}, label %[[BODY:[0-9]+]], label %[[LOOP_EXIT:[0-9]+]]
+// CHECK: [[BODY]]:
+// CHECK: %[[DYNAMIC_ALLOCATOR:.*]] = add i64 %{{.*}}, %{{.*}}
+// CHECK: %[[ALLOCATOR_HANDLE:.*]] = inttoptr i64 %[[DYNAMIC_ALLOCATOR]] to ptr
+// CHECK: %[[LOOP_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr %[[ALLOCATOR_HANDLE]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 {{.*}}, ptr %[[LOOP_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[LOOP_ALLOC]], ptr %[[ALLOCATOR_HANDLE]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: br label %[[LOOP]]
+// CHECK: [[LOOP_EXIT]]:
+// CHECK: ret void
+
+//--- device.mlir
+
+omp.private {type = private} @device.private : i32
+
+llvm.func @scope_allocator_device() {
+ omp.target kernel_type(generic) {
+ %allocator = llvm.mlir.constant(0 : i64) : i64
+ %x = llvm.alloca %allocator x i32 : (i64) -> !llvm.ptr
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_alignments([64]) allocate_private_indices([0])
+ private(@device.private %x -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ omp.terminator
+ }
+ llvm.return
+}
+
+// DEVICE-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// DEVICE: allocate clause in an OpenMP device context is not supported
+// DEVICE: LLVM Translation failed for operation: omp.scope
+// DEVICE-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
diff --git a/mlir/test/Target/LLVMIR/openmp-todo.mlir b/mlir/test/Target/LLVMIR/openmp-todo.mlir
index 635665c653f8d..391cd8d61d091 100644
--- a/mlir/test/Target/LLVMIR/openmp-todo.mlir
+++ b/mlir/test/Target/LLVMIR/openmp-todo.mlir
@@ -77,17 +77,6 @@ llvm.func @sections_allocate(%x : !llvm.ptr) {
// -----
-llvm.func @scope_allocate(%x : !llvm.ptr) {
- // expected-error at below {{not yet implemented: Unhandled clause allocate in omp.scope operation}}
- // expected-error at below {{LLVM Translation failed for operation: omp.scope}}
- omp.scope allocate(%x : !llvm.ptr -> %x : !llvm.ptr) {
- omp.terminator
- }
- llvm.return
-}
-
-// -----
-
omp.private {type = private} @x.privatizer : i32 init {
^bb0(%mold: !llvm.ptr, %private: !llvm.ptr):
%c0 = llvm.mlir.constant(0 : i32) : i32
More information about the flang-commits
mailing list