[flang-commits] [flang] b1f6690 - [flang][OpenMP] Support allocate clauses on scope constructs (#221229)
via flang-commits
flang-commits at lists.llvm.org
Sun Sep 27 20:41:25 PDT 2026
Author: Sairudra More
Date: 2026-09-28T09:11:18+05:30
New Revision: b1f66905515904d048c9c53e92b1c0d2423aca19
URL: https://github.com/llvm/llvm-project/commit/b1f66905515904d048c9c53e92b1c0d2423aca19
DIFF: https://github.com/llvm/llvm-project/commit/b1f66905515904d048c9c53e92b1c0d2423aca19.diff
LOG: [flang][OpenMP] Support allocate clauses on scope constructs (#221229)
Flang already lowers OpenMP 5.2 `allocate` clauses on `scope` constructs
to `omp.scope`, but LLVM IR translation previously rejected them.
This change maps allocation operands to their corresponding private or
firstprivate slots and reuses the existing allocator-backed
private-storage and OpenMPIRBuilder allocation/free paths.
Support is currently limited to host-side fixed-size intrinsic scalar
private and firstprivate variables, including SAVE variables and
individually listed COMMON-block members. Arrays, descriptors, pointers,
allocatables, derived types, named COMMON-block list items, and
target/device allocation remain unsupported.
Added:
flang/test/Integration/OpenMP/allocate-clause-equivalence.f90
flang/test/Integration/OpenMP/scope-allocate-clause.f90
flang/test/Lower/OpenMP/Todo/scope-allocate-clause-unsupported.f90
mlir/test/Target/LLVMIR/openmp-scope-allocate-clause.mlir
Modified:
flang/docs/OpenMPSupport.md
flang/lib/Lower/OpenMP/OpenMP.cpp
flang/test/Lower/OpenMP/Todo/allocate-clause-unsupported.f90
flang/test/Lower/OpenMP/scope.f90
flang/test/Lower/OpenMP/target-scope.f90
mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
mlir/test/Dialect/OpenMP/invalid.mlir
mlir/test/Dialect/OpenMP/ops.mlir
mlir/test/Target/LLVMIR/openmp-allocate-clause.mlir
mlir/test/Target/LLVMIR/openmp-todo.mlir
Removed:
################################################################################
diff --git a/flang/docs/OpenMPSupport.md b/flang/docs/OpenMPSupport.md
index abd6b1b536cb3..475add18a0104 100644
--- a/flang/docs/OpenMPSupport.md
+++ b/flang/docs/OpenMPSupport.md
@@ -221,7 +221,7 @@ Parser/Semantics, MLIR, Lowering, or the OpenMPIRBuilder.
| otherwise clause on metadirectives | <span class="good">done</span> | | Lowering handles `OmpClause::Otherwise` as the fallback variant in `genMetadirective` (`flang/lib/Lower/OpenMP/OpenMP.cpp`); the legacy `default` clause spelling is also handled. | [llvm/llvm-project#194402](https://github.com/llvm/llvm-project/pull/194402), [llvm/llvm-project#194424](https://github.com/llvm/llvm-project/pull/194424) |
| doacross with omp_cur_iteration | <span class="progress">in progress</span> | dreachem | Implement parser/semantics validation for `omp_cur_iteration` placement and lowering of doacross dependence tokens, then add semantics+lowering tests for source/sink combinations. | |
| implicit map type for target enter and exit data | <span class="good">done</span> | | Target enter data applies implicit `to` mapping; target exit data applies implicit `from` mapping. Lowering via `getImplicitMapTypeAndKind` and MapInfoFinalization pass. Comprehensive test coverage in `target-enter-data-default-openmp52.f90` for OpenMP 5.2+ including allocatable descriptor handling. | [llvm/llvm-project#174665](https://github.com/llvm/llvm-project/pull/174665), [llvm/llvm-project#193851](https://github.com/llvm/llvm-project/pull/193851) |
-| allocate and firstprivate on scope directive | <span class="part">partial</span> | | Firstprivate on scope is fully supported and tested (`scope.f90`). Allocate clause on scope is parsed/lowered to MLIR but not yet implemented in LLVM IR translation layer (checkImplementationStatus in OpenMPToLLVMIRTranslation.cpp blocks it). | [llvm/llvm-project#193098](https://github.com/llvm/llvm-project/pull/193098) |
+| allocate and firstprivate on scope directive | <span class="part">partial</span> | | Firstprivate on scope is fully supported and tested (`scope.f90`). Host-side allocate clauses on scope use allocator-backed storage for fixed-size intrinsic scalar PRIVATE/FIRSTPRIVATE items with omitted/default/explicit allocator and ALIGN (`scope-allocate-clause.f90`, `openmp-scope-allocate-clause.mlir`), including SAVE entities and individually listed common-block members. Arrays, descriptors, pointers, allocatables, derived types, named common-block list items, and target/device scope allocation remain unsupported. | [llvm/llvm-project#193098](https://github.com/llvm/llvm-project/pull/193098) |
| loop consistency changes for order clause | <span class="none">unclaimed</span> | | Extend semantic loop-consistency checks for updated ORDER rules and add diagnostics tests for invalid nest/ordering combinations. | |
| keep original base pointer on map without matched candidate | <span class="none">unclaimed</span> | | Update map finalization so unmatched candidates preserve original base-pointer mapping semantics; add lowering tests for pointer-member mapping regressions. | |
| pure procedure support for certain directives | <span class="good">done</span> | ShashwathiNavada | Semantics enforce that metadirective, assumption directives (ASSUME/ASSUMES), NOTHING, ERROR, and the loop-transforming constructs TILE/UNROLL are permitted in a Fortran PURE procedure, in addition to the baseline of SIMD and declarative directives; other directives are rejected with a version-aware diagnostic. | [llvm/llvm-project#212676](https://github.com/llvm/llvm-project/pull/212676) |
@@ -280,4 +280,3 @@ The OpenMP spec requires intrinsic- or pointer-assignments, which include (as pe
- `expr equalop x` is an allowed condition in ATOMIC UPDATE COMPARE. [1]
[1] Code generation for ATOMIC UPDATE COMPARE is not implemented yet.
-
diff --git a/flang/lib/Lower/OpenMP/OpenMP.cpp b/flang/lib/Lower/OpenMP/OpenMP.cpp
index a815cb2298719..455151458f398 100644
--- a/flang/lib/Lower/OpenMP/OpenMP.cpp
+++ b/flang/lib/Lower/OpenMP/OpenMP.cpp
@@ -2612,7 +2612,7 @@ static void genScopeClauses(lower::AbstractConverter &converter,
mlir::omp::ScopeOperands &clauseOps,
llvm::SmallVectorImpl<Object> &reductionObjects) {
ClauseProcessor cp(converter, semaCtx, clauses);
- cp.processAllocate(clauseOps);
+ cp.processAllocate(clauseOps, /*supportAlignment=*/true);
cp.processNowait(clauseOps);
cp.processReduction(loc, clauseOps, reductionObjects);
}
@@ -3525,6 +3525,83 @@ genOrderedRegionOp(lower::AbstractConverter &converter, lower::SymMap &symTable,
queue, item, clauseOps);
}
+/// Private operand order follows privatization expansion rather than the
+/// ALLOCATE clause's list order.
+template <typename ClauseOpsT>
+static void mapAllocateClauseToPrivateSlots(mlir::Location loc,
+ const List<Clause> &clauses,
+ const ObjectEntryBlockArgs &args,
+ ClauseOpsT &clauseOps) {
+ if (clauseOps.allocateVars.empty())
+ return;
+
+ llvm::DenseMap<const semantics::Symbol *, int64_t> privateSlots;
+ int64_t privateSlot = 0;
+ auto addPrivateSlot = [&](const semantics::Symbol &symbol) {
+ if (!privateSlots.try_emplace(&symbol.GetUltimate(), privateSlot).second)
+ fir::emitFatalError(
+ loc, "symbol with multiple private storage slots on one construct");
+ ++privateSlot;
+ };
+ for (const Object &object : args.priv.objects) {
+ const semantics::Symbol *symbol = object.sym();
+ if (!symbol)
+ fir::emitFatalError(loc, "private item without a semantic symbol");
+ // A privatized common block contributes one private operand per member,
+ // so slot numbering must follow the same expansion.
+ if (const auto *commonDetails =
+ symbol->detailsIf<semantics::CommonBlockDetails>()) {
+ for (const auto &member : commonDetails->objects())
+ addPrivateSlot(*member);
+ } else {
+ addPrivateSlot(*symbol);
+ }
+ }
+
+ llvm::DenseSet<const semantics::Symbol *> allocateSymbols;
+ for (const Clause &clause : clauses) {
+ if (clause.id != llvm::omp::Clause::OMPC_allocate)
+ continue;
+ const auto &allocate = std::get<clause::Allocate>(clause.u);
+ const auto &objects = std::get<ObjectList>(allocate.t);
+ for (const Object &object : objects) {
+ const semantics::Symbol *symbol = object.sym();
+ if (!symbol)
+ fir::emitFatalError(loc,
+ "ALLOCATE clause item without a semantic symbol");
+ const semantics::Symbol *ultimate = &symbol->GetUltimate();
+ if (!allocateSymbols.insert(ultimate).second)
+ TODO(loc, "ALLOCATE clause item appears more than once");
+
+ auto privateSlot = privateSlots.find(ultimate);
+ if (privateSlot == privateSlots.end())
+ fir::emitFatalError(
+ loc, "ALLOCATE clause item without private storage slot");
+
+ auto type = evaluate::DynamicType::From(*ultimate);
+ bool supportedDataSharing =
+ symbol->test(semantics::Symbol::Flag::OmpPrivate) ||
+ symbol->test(semantics::Symbol::Flag::OmpFirstPrivate);
+ bool supportedType = ultimate->Rank() == 0 &&
+ !semantics::IsAllocatableOrPointer(*ultimate) &&
+ type &&
+ type->category() != common::TypeCategory::Derived &&
+ !type->RequiresDescriptor() &&
+ !type->HasDeferredOrAssumedTypeParameter();
+ if (!supportedDataSharing || !supportedType)
+ TODO(loc,
+ "ALLOCATE clause currently supports only fixed-size intrinsic "
+ "scalar PRIVATE or FIRSTPRIVATE items");
+
+ clauseOps.allocatePrivateIndices.push_back(privateSlot->second);
+ }
+ }
+
+ if (clauseOps.allocatePrivateIndices.size() != clauseOps.allocateVars.size())
+ fir::emitFatalError(loc,
+ "incomplete ALLOCATE clause private storage mapping");
+}
+
static mlir::omp::ParallelOp
genParallelOp(lower::AbstractConverter &converter, lower::SymMap &symTable,
semantics::SemanticsContext &semaCtx,
@@ -3536,74 +3613,7 @@ genParallelOp(lower::AbstractConverter &converter, lower::SymMap &symTable,
assert((!enableDelayedPrivatization || dsp) &&
"expected valid DataSharingProcessor");
- if (!clauseOps.allocateVars.empty()) {
- llvm::DenseMap<const semantics::Symbol *, int64_t> privateSlots;
- int64_t privateSlot = 0;
- auto addPrivateSlot = [&](const semantics::Symbol &symbol) {
- if (!privateSlots.try_emplace(&symbol.GetUltimate(), privateSlot).second)
- fir::emitFatalError(
- loc, "symbol with multiple private storage slots on one construct");
- ++privateSlot;
- };
- for (const Object &object : args.priv.objects) {
- const semantics::Symbol *symbol = object.sym();
- if (!symbol)
- fir::emitFatalError(loc, "private item without a semantic symbol");
- // A privatized common block contributes one private operand per member,
- // so slot numbering must follow the same expansion.
- if (const auto *commonDetails =
- symbol->detailsIf<semantics::CommonBlockDetails>()) {
- for (const auto &member : commonDetails->objects())
- addPrivateSlot(*member);
- } else {
- addPrivateSlot(*symbol);
- }
- }
-
- llvm::DenseSet<const semantics::Symbol *> allocateSymbols;
- for (const Clause &clause : item->clauses) {
- if (clause.id != llvm::omp::Clause::OMPC_allocate)
- continue;
- const auto &allocate = std::get<clause::Allocate>(clause.u);
- const auto &objects = std::get<ObjectList>(allocate.t);
- for (const Object &object : objects) {
- const semantics::Symbol *symbol = object.sym();
- if (!symbol)
- fir::emitFatalError(loc,
- "ALLOCATE clause item without a semantic symbol");
- const semantics::Symbol *ultimate = &symbol->GetUltimate();
- if (!allocateSymbols.insert(ultimate).second)
- TODO(loc, "ALLOCATE clause item appears more than once");
-
- auto privateSlot = privateSlots.find(ultimate);
- if (privateSlot == privateSlots.end())
- fir::emitFatalError(
- loc, "ALLOCATE clause item without private storage slot");
-
- auto type = evaluate::DynamicType::From(*ultimate);
- bool supportedDataSharing =
- symbol->test(semantics::Symbol::Flag::OmpPrivate) ||
- symbol->test(semantics::Symbol::Flag::OmpFirstPrivate);
- bool supportedType =
- ultimate->Rank() == 0 &&
- !semantics::IsAllocatableOrPointer(*ultimate) && type &&
- type->category() != common::TypeCategory::Derived &&
- !type->RequiresDescriptor() &&
- !type->HasDeferredOrAssumedTypeParameter();
- if (!supportedDataSharing || !supportedType)
- TODO(loc,
- "ALLOCATE clause currently supports only fixed-size intrinsic "
- "scalar PRIVATE or FIRSTPRIVATE items");
-
- clauseOps.allocatePrivateIndices.push_back(privateSlot->second);
- }
- }
-
- if (clauseOps.allocatePrivateIndices.size() !=
- clauseOps.allocateVars.size())
- fir::emitFatalError(loc,
- "incomplete ALLOCATE clause private storage mapping");
- }
+ mapAllocateClauseToPrivateSlots(loc, item->clauses, args, clauseOps);
OpWithBodyGenInfo genInfo =
OpWithBodyGenInfo(converter, symTable, semaCtx, loc, eval,
@@ -3964,6 +3974,8 @@ genScopeOp(lower::AbstractConverter &converter, lower::SymMap &symTable,
args.reduction.objects = reductionObjects;
args.reduction.vars = clauseOps.reductionVars;
+ mapAllocateClauseToPrivateSlots(loc, item->clauses, args, clauseOps);
+
return genOpWithBody<mlir::omp::ScopeOp>(
OpWithBodyGenInfo(converter, symTable, semaCtx, loc, eval,
llvm::omp::Directive::OMPD_scope)
diff --git a/flang/test/Integration/OpenMP/allocate-clause-equivalence.f90 b/flang/test/Integration/OpenMP/allocate-clause-equivalence.f90
new file mode 100644
index 0000000000000..2ea7706f75905
--- /dev/null
+++ b/flang/test/Integration/OpenMP/allocate-clause-equivalence.f90
@@ -0,0 +1,47 @@
+!===----------------------------------------------------------------------===!
+! This directory can be used to add Integration tests involving multiple
+! stages of the compiler (for eg. from Fortran to LLVM IR). It should not
+! contain executable tests. We should only add tests here sparingly and only
+! if there is no other way to test. Repeat this message in each test that is
+! added to this directory and sub-directories.
+!===----------------------------------------------------------------------===!
+
+! RUN: %flang_fc1 -fsyntax-only %openmp_flags -fopenmp-version=52 %s
+! RUN: %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %s | FileCheck %s --check-prefix=HLFIR
+! RUN: %flang_fc1 -emit-llvm %openmp_flags -fopenmp-version=52 -o - %s | FileCheck %s --check-prefix=LLVM
+
+subroutine scope_equivalence()
+ integer :: x, y
+ equivalence (x, y)
+ !$omp scope private(x, y) allocate(x, y)
+ x = 1
+ call consume(x)
+ y = 2
+ call consume(y)
+ !$omp end scope
+end subroutine
+
+! HLFIR-LABEL: func.func @_QPscope_equivalence
+! HLFIR: omp.scope allocate(
+! HLFIR-SAME: allocate_private_indices([0, 1])
+! LLVM-LABEL: define void @scope_equivalence_
+! LLVM: call ptr @__kmpc_alloc
+! LLVM: call ptr @__kmpc_alloc
+
+subroutine parallel_equivalence()
+ integer :: x, y
+ equivalence (x, y)
+ !$omp parallel private(x, y) allocate(x, y)
+ x = 1
+ call consume(x)
+ y = 2
+ call consume(y)
+ !$omp end parallel
+end subroutine
+
+! HLFIR-LABEL: func.func @_QPparallel_equivalence
+! HLFIR: omp.parallel allocate(
+! HLFIR-SAME: allocate_private_indices([0, 1])
+! LLVM-LABEL: define internal void @parallel_equivalence_..omp_par
+! LLVM: call ptr @__kmpc_alloc
+! LLVM: call ptr @__kmpc_alloc
diff --git a/flang/test/Integration/OpenMP/scope-allocate-clause.f90 b/flang/test/Integration/OpenMP/scope-allocate-clause.f90
new file mode 100644
index 0000000000000..50bd1ab288d8d
--- /dev/null
+++ b/flang/test/Integration/OpenMP/scope-allocate-clause.f90
@@ -0,0 +1,225 @@
+!===----------------------------------------------------------------------===!
+! This directory can be used to add Integration tests involving multiple
+! stages of the compiler (for eg. from Fortran to LLVM IR). It should not
+! contain executable tests. We should only add tests here sparingly and only
+! if there is no other way to test. Repeat this message in each test that is
+! added to this directory and sub-directories.
+!===----------------------------------------------------------------------===!
+
+! This test checks lowering of the ALLOCATE clause on the OpenMP SCOPE
+! construct: PRIVATE with omitted/default allocator, PRIVATE with an explicit
+! allocator, FIRSTPRIVATE with allocator-backed storage, SAVE and individual
+! COMMON members, ALIGN, and an allocation-list order that
diff ers from the
+! private-list order.
+
+! RUN: %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %s 2>&1 | FileCheck %s
+! RUN: %flang_fc1 -emit-llvm %openmp_flags -fopenmp-version=52 -o - %s 2>&1 | FileCheck %s --check-prefix=LLVM
+! RUN: not %flang_fc1 -fsyntax-only %openmp_flags -fopenmp-version=51 %s 2>&1 | FileCheck %s --check-prefix=VERSION51
+
+! VERSION51: error: {{.*}}ALLOCATE
+
+subroutine scope_allocator_omitted(x, y)
+ integer :: x, y
+ !$omp scope private(x, y) allocate(x)
+ x = 1
+ y = 2
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_omitted
+! CHECK: %[[NULL_ALLOC:.*]] = arith.constant 0 : i32
+! CHECK: omp.scope allocate(%[[NULL_ALLOC]] : i32 -> %[[X:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[X]]#0 -> %[[X_PRIVATE:.*]], {{.*}} -> %[[Y_PRIVATE:.*]] : !fir.ref<i32>, !fir.ref<i32>) {
+
+subroutine scope_allocator_explicit(x, allocator)
+ use iso_c_binding, only : c_intptr_t
+ integer :: x
+ integer(c_intptr_t), intent(in) :: allocator
+ !$omp scope private(x) allocate(allocator(allocator): x)
+ x = 1
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_explicit
+! CHECK: %[[ALLOCATOR:.*]] = fir.load %{{.*}} : !fir.ref<i64>
+! CHECK: omp.scope allocate(%[[ALLOCATOR]] : i64 -> %[[X:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[X]]#0 -> {{.*}} : !fir.ref<i32>) {
+
+subroutine scope_allocator_firstprivate(x)
+ integer :: x
+ !$omp scope firstprivate(x) allocate(x)
+ x = x + 1
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_firstprivate
+! CHECK: omp.scope allocate({{.*}} : i32 -> %[[X:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[X]]#0 -> %[[X_PRIVATE:.*]] : !fir.ref<i32>) {
+! CHECK: %[[X_DECL:.*]]:2 = hlfir.declare %[[X_PRIVATE]]
+! CHECK: hlfir.assign %{{.*}} to %[[X_DECL]]#0
+
+subroutine scope_allocator_align(x, y)
+ integer :: x, y
+ !$omp scope private(x, y) allocate(x) allocate(align(64): y)
+ x = 1
+ y = 2
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_align
+! CHECK: omp.scope allocate(
+! CHECK-SAME: allocate_alignments([0, 64]) allocate_private_indices([0, 1])
+! CHECK-SAME: private(
+
+subroutine scope_allocator_order(x, y, z, allocator_xy, allocator_z)
+ use iso_c_binding, only : c_intptr_t
+ integer :: x, y, z
+ integer(c_intptr_t), intent(in) :: allocator_xy, allocator_z
+ !$omp scope private(x, y, z) &
+ !$omp& allocate(allocator(allocator_xy): y, x) &
+ !$omp& allocate(allocator(allocator_z): z)
+ x = 1
+ y = 2
+ z = 3
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_order
+! CHECK: omp.scope allocate(
+! CHECK-SAME: allocate_private_indices([1, 0, 2])
+! CHECK-SAME: private(
+
+subroutine scope_allocator_dynamic(x, allocators, n)
+ use iso_c_binding, only : c_intptr_t
+ integer, intent(in) :: n
+ integer(c_intptr_t), intent(in) :: allocators(n)
+ integer :: x, i
+ do i = 1, n
+ !$omp scope firstprivate(x) allocate(allocator(allocators(i)): x)
+ x = x + i
+ !$omp end scope
+ end do
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_dynamic
+! CHECK: fir.do_loop
+! CHECK: %[[DYNAMIC_ALLOCATOR:.*]] = fir.load %{{.*}} : !fir.ref<i64>
+! CHECK: omp.scope allocate(%[[DYNAMIC_ALLOCATOR]] : i64 -> %[[X:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[X]]#0 -> {{.*}} : !fir.ref<i32>) {
+
+! LLVM-LABEL: define void @scope_allocator_dynamic_
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM: omp.region.after_alloca:
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM: br label %[[LOOP:[0-9]+]]
+! LLVM: [[LOOP]]:
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM: br i1 {{.*}}, label %[[BODY:[0-9]+]], label %[[EXIT:[0-9]+]]
+! LLVM: [[BODY]]:
+! LLVM: %[[DYNAMIC_ALLOCATOR:.*]] = load i64
+! LLVM: %[[ALLOCATOR_HANDLE:.*]] = inttoptr i64 %[[DYNAMIC_ALLOCATOR]] to ptr
+! LLVM: %[[ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr %[[ALLOCATOR_HANDLE]])
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM-NOT: call void @__kmpc_free
+! LLVM: store i32 {{.*}}, ptr %[[ALLOC]]
+! LLVM-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+! LLVM-NOT: call void @__kmpc_free
+! LLVM: call void @__kmpc_free({{.*}}, ptr %[[ALLOC]], ptr %[[ALLOCATOR_HANDLE]])
+! LLVM-NOT: call void @__kmpc_free
+! LLVM: br label %[[LOOP]]
+! LLVM: [[EXIT]]:
+! LLVM: ret void
+
+subroutine scope_allocator_private_save()
+ integer, save :: x
+ !$omp scope private(x) allocate(x)
+ x = 11
+ call consume(x)
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_private_save
+! CHECK: omp.scope allocate({{.*}} -> %[[SAVE:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[SAVE]]#0 -> %[[SAVE_PRIVATE:.*]] : !fir.ref<i32>) {
+! CHECK: %[[SAVE_DECL:.*]]:2 = hlfir.declare %[[SAVE_PRIVATE]]
+! CHECK: hlfir.assign %c11_i32 to %[[SAVE_DECL]]#0
+! CHECK: fir.call @_QPconsume(%[[SAVE_DECL]]#0)
+
+! LLVM-LABEL: define void @scope_allocator_private_save_
+! LLVM: %[[SAVE_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+! LLVM: store i32 11, ptr %[[SAVE_ALLOC]]
+! LLVM: call void @consume_(ptr %[[SAVE_ALLOC]])
+! LLVM: call void @__kmpc_free({{.*}}, ptr %[[SAVE_ALLOC]], ptr null)
+
+subroutine scope_allocator_firstprivate_save()
+ integer, save :: x = 23
+ !$omp scope firstprivate(x) allocate(x)
+ call consume(x)
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_firstprivate_save
+! CHECK: omp.scope allocate({{.*}} -> %[[SAVE:.*]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[SAVE]]#0 -> %[[SAVE_PRIVATE:.*]] : !fir.ref<i32>) {
+! CHECK: %[[SAVE_DECL:.*]]:2 = hlfir.declare %[[SAVE_PRIVATE]]
+! CHECK: fir.call @_QPconsume(%[[SAVE_DECL]]#0)
+
+! LLVM-LABEL: define void @scope_allocator_firstprivate_save_
+! LLVM: %[[SAVE_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+! LLVM: %[[SAVE_VALUE:.*]] = load i32, ptr @_QFscope_allocator_firstprivate_saveEx
+! LLVM: store i32 %[[SAVE_VALUE]], ptr %[[SAVE_ALLOC]]
+! LLVM: call void @consume_(ptr %[[SAVE_ALLOC]])
+! LLVM: call void @__kmpc_free({{.*}}, ptr %[[SAVE_ALLOC]], ptr null)
+
+subroutine scope_allocator_private_common()
+ integer :: padding, x
+ common /scope_private_common/ padding, x
+ !$omp scope private(x) allocate(x)
+ x = 17
+ call consume(x)
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_private_common
+! CHECK: %[[COMMON:.*]] = fir.address_of(@scope_private_common_) : !fir.ref<!fir.array<8xi8>>
+! CHECK: %[[OFFSET:.*]] = arith.constant 4 : index
+! CHECK: %[[ADDRESS:.*]] = fir.coordinate_of %[[COMMON]], %[[OFFSET]]
+! CHECK: %[[MEMBER_ADDRESS:.*]] = fir.convert %[[ADDRESS]] : (!fir.ref<i8>) -> !fir.ref<i32>
+! CHECK: %[[MEMBER:.*]]:2 = hlfir.declare %[[MEMBER_ADDRESS]] storage(%[[COMMON]][4])
+! CHECK: omp.scope allocate({{.*}} -> %[[MEMBER]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[MEMBER]]#0 -> %[[PRIVATE:.*]] : !fir.ref<i32>) {
+! CHECK: %[[PRIVATE_DECL:.*]]:2 = hlfir.declare %[[PRIVATE]]
+! CHECK: hlfir.assign %c17_i32 to %[[PRIVATE_DECL]]#0
+! CHECK: fir.call @_QPconsume(%[[PRIVATE_DECL]]#0)
+
+! LLVM-LABEL: define void @scope_allocator_private_common_
+! LLVM: %[[COMMON_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+! LLVM: store i32 17, ptr %[[COMMON_ALLOC]]
+! LLVM: call void @consume_(ptr %[[COMMON_ALLOC]])
+! LLVM: call void @__kmpc_free({{.*}}, ptr %[[COMMON_ALLOC]], ptr null)
+
+subroutine scope_allocator_firstprivate_common()
+ integer :: padding, x
+ common /scope_firstprivate_common/ padding, x
+ !$omp scope firstprivate(x) allocate(x)
+ call consume(x)
+ !$omp end scope
+end subroutine
+
+! CHECK-LABEL: func.func @_QPscope_allocator_firstprivate_common
+! CHECK: %[[COMMON:.*]] = fir.address_of(@scope_firstprivate_common_) : !fir.ref<!fir.array<8xi8>>
+! CHECK: %[[OFFSET:.*]] = arith.constant 4 : index
+! CHECK: %[[ADDRESS:.*]] = fir.coordinate_of %[[COMMON]], %[[OFFSET]]
+! CHECK: %[[MEMBER_ADDRESS:.*]] = fir.convert %[[ADDRESS]] : (!fir.ref<i8>) -> !fir.ref<i32>
+! CHECK: %[[MEMBER:.*]]:2 = hlfir.declare %[[MEMBER_ADDRESS]] storage(%[[COMMON]][4])
+! CHECK: omp.scope allocate({{.*}} -> %[[MEMBER]]#0 : !fir.ref<i32>) allocate_private_indices([0])
+! CHECK-SAME: private({{.*}} %[[MEMBER]]#0 -> %[[PRIVATE:.*]] : !fir.ref<i32>) {
+! CHECK: %[[PRIVATE_DECL:.*]]:2 = hlfir.declare %[[PRIVATE]]
+! CHECK: fir.call @_QPconsume(%[[PRIVATE_DECL]]#0)
+
+! LLVM-LABEL: define void @scope_allocator_firstprivate_common_
+! LLVM: %[[COMMON_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+! LLVM: %[[COMMON_VALUE:.*]] = load i32, ptr getelementptr inbounds{{.*}} @scope_firstprivate_common_{{.*}} i64 4
+! LLVM: store i32 %[[COMMON_VALUE]], ptr %[[COMMON_ALLOC]]
+! LLVM: call void @consume_(ptr %[[COMMON_ALLOC]])
+! LLVM: call void @__kmpc_free({{.*}}, ptr %[[COMMON_ALLOC]], ptr null)
diff --git a/flang/test/Lower/OpenMP/Todo/allocate-clause-unsupported.f90 b/flang/test/Lower/OpenMP/Todo/allocate-clause-unsupported.f90
index 72e83c6967642..5b6ccd470daa5 100644
--- a/flang/test/Lower/OpenMP/Todo/allocate-clause-unsupported.f90
+++ b/flang/test/Lower/OpenMP/Todo/allocate-clause-unsupported.f90
@@ -1,5 +1,6 @@
! RUN: split-file %s %t
! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=51 -o - %t/duplicate.f90 2>&1 | FileCheck %s --check-prefix=DUPLICATE
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/duplicate-scope.f90 2>&1 | FileCheck %s --check-prefix=DUPLICATE-SCOPE
! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=51 -o - %t/array.f90 2>&1 | FileCheck %s --check-prefix=ARRAY
! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=51 -o - %t/derived.f90 2>&1 | FileCheck %s --check-prefix=DERIVED
! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=51 -o - %t/pointer.f90 2>&1 | FileCheck %s --check-prefix=POINTER
@@ -8,6 +9,7 @@
! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=51 -o - %t/non-parallel.f90 2>&1 | FileCheck %s --check-prefix=NON-PARALLEL
! DUPLICATE: not yet implemented: ALLOCATE clause item appears more than once
+! DUPLICATE-SCOPE: not yet implemented: ALLOCATE clause item appears more than once
! ARRAY: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
! DERIVED: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
! POINTER: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
@@ -23,6 +25,14 @@ subroutine duplicate(x)
!$omp end parallel
end subroutine
+!--- duplicate-scope.f90
+subroutine duplicate_scope(x)
+ integer :: x
+ !$omp scope private(x) allocate(x) allocate(x)
+ x = 1
+ !$omp end scope
+end subroutine
+
!--- pointer.f90
subroutine pointer(x)
integer, pointer :: x
diff --git a/flang/test/Lower/OpenMP/Todo/scope-allocate-clause-unsupported.f90 b/flang/test/Lower/OpenMP/Todo/scope-allocate-clause-unsupported.f90
new file mode 100644
index 0000000000000..4763c6643ee11
--- /dev/null
+++ b/flang/test/Lower/OpenMP/Todo/scope-allocate-clause-unsupported.f90
@@ -0,0 +1,55 @@
+! RUN: split-file %s %t
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/array.f90 2>&1 | FileCheck %s --check-prefix=ARRAY
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/derived.f90 2>&1 | FileCheck %s --check-prefix=DERIVED
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/pointer.f90 2>&1 | FileCheck %s --check-prefix=POINTER
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/allocatable.f90 2>&1 | FileCheck %s --check-prefix=ALLOCATABLE
+! RUN: %not_todo_cmd %flang_fc1 -emit-hlfir %openmp_flags -fopenmp-version=52 -o - %t/assumed-length.f90 2>&1 | FileCheck %s --check-prefix=ASSUMED-LENGTH
+
+! ARRAY: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! DERIVED: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! POINTER: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! ALLOCATABLE: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+! ASSUMED-LENGTH: not yet implemented: ALLOCATE clause currently supports only fixed-size intrinsic scalar PRIVATE or FIRSTPRIVATE items
+
+!--- array.f90
+subroutine array(x)
+ integer :: x(4)
+ !$omp scope private(x) allocate(align(64): x)
+ x = 1
+ !$omp end scope
+end subroutine
+
+!--- derived.f90
+subroutine derived()
+ type t
+ integer :: value
+ end type
+ type(t) :: x
+ !$omp scope private(x) allocate(x)
+ x%value = 1
+ !$omp end scope
+end subroutine
+
+!--- pointer.f90
+subroutine pointer(x)
+ integer, pointer :: x
+ !$omp scope private(x) allocate(x)
+ x = 1
+ !$omp end scope
+end subroutine
+
+!--- allocatable.f90
+subroutine allocatable(x)
+ integer, allocatable :: x
+ !$omp scope private(x) allocate(x)
+ x = 1
+ !$omp end scope
+end subroutine
+
+!--- assumed-length.f90
+subroutine assumed_length(x)
+ character(*) :: x
+ !$omp scope private(x) allocate(x)
+ x = "test"
+ !$omp end scope
+end subroutine
diff --git a/flang/test/Lower/OpenMP/scope.f90 b/flang/test/Lower/OpenMP/scope.f90
index baa5ed493052b..2593bf5b6253c 100644
--- a/flang/test/Lower/OpenMP/scope.f90
+++ b/flang/test/Lower/OpenMP/scope.f90
@@ -103,7 +103,7 @@ subroutine omp_scope_allocate()
! CHECK: hlfir.declare %{{.*}} {uniq_name = "_QFomp_scope_allocateEi"}
i = 0
- ! CHECK: omp.scope allocate(%{{.*}} : i32 -> %{{.*}}#0 : !fir.ref<i32>) private(@_QFomp_scope_allocateEi_private_i32 %{{.*}}#0 -> %[[APRIV:.*]] : !fir.ref<i32>) {
+ ! CHECK: omp.scope allocate(%{{.*}} : i32 -> %{{.*}}#0 : !fir.ref<i32>) allocate_private_indices([0]) private(@_QFomp_scope_allocateEi_private_i32 %{{.*}}#0 -> %[[APRIV:.*]] : !fir.ref<i32>) {
! CHECK: %[[ADECL:.*]]:2 = hlfir.declare %[[APRIV]] {uniq_name = "_QFomp_scope_allocateEi"}
!$omp scope private(i) allocate(i)
! CHECK: hlfir.assign %{{.*}} to %[[ADECL]]#0 : i32, !fir.ref<i32>
diff --git a/flang/test/Lower/OpenMP/target-scope.f90 b/flang/test/Lower/OpenMP/target-scope.f90
index de3bee3bdcacb..0684c529c96b9 100644
--- a/flang/test/Lower/OpenMP/target-scope.f90
+++ b/flang/test/Lower/OpenMP/target-scope.f90
@@ -123,7 +123,7 @@ subroutine target_scope_allocate()
!$omp target
! CHECK: omp.target kernel_type(generic) map_entries(%{{.*}} -> %[[IARG:.*]] : !fir.ref<i32>) {
! CHECK: hlfir.declare %[[IARG]] {uniq_name = "_QFtarget_scope_allocateEi"}
- ! CHECK: omp.scope allocate(%{{.*}} : i32 -> %{{.*}}#0 : !fir.ref<i32>) private(@_QFtarget_scope_allocateEi_private_i32 %{{.*}}#0 -> %[[APRIV:.*]] : !fir.ref<i32>) {
+ ! CHECK: omp.scope allocate(%{{.*}} : i32 -> %{{.*}}#0 : !fir.ref<i32>) allocate_private_indices([0]) private(@_QFtarget_scope_allocateEi_private_i32 %{{.*}}#0 -> %[[APRIV:.*]] : !fir.ref<i32>) {
! CHECK: %[[ADECL:.*]]:2 = hlfir.declare %[[APRIV]] {uniq_name = "_QFtarget_scope_allocateEi"}
!$omp scope private(i) allocate(i)
! CHECK: hlfir.assign %{{.*}} to %[[ADECL]]#0 : i32, !fir.ref<i32>
diff --git a/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp b/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
index ed144f7b28983..3822ef3f44046 100644
--- a/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
+++ b/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
@@ -723,6 +723,10 @@ static LogicalResult verifyAllocateClause(
<< "type mismatch between allocate variable and private variable "
"at index "
<< privateIndex;
+ if (allocateVar != privateVar)
+ return op->emitError()
+ << "allocate variable does not match private variable at index "
+ << privateIndex;
if (!privateSyms ||
static_cast<uint64_t>(privateIndex) >= privateSyms.size())
@@ -3276,7 +3280,8 @@ LogicalResult ScopeOp::verify() {
if (failed(verifyAllocateClause(
getOperation(), getAllocateVars(), getAllocatorVars(),
getAllocateAlignmentsAttr(), getAllocatePrivateIndicesAttr(),
- getPrivateVars(), getPrivateSymsAttr())))
+ getPrivateVars(), getPrivateSymsAttr(),
+ /*requirePrivateIndices=*/true)))
return failure();
if (failed(verifyPrivateVarList(*this)))
diff --git a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
index 9f137c102a9c2..1cfac8cf60be6 100644
--- a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
@@ -548,10 +548,7 @@ static LogicalResult checkImplementationStatus(Operation &op) {
checkPrivate(op, result);
checkReduction(op, result);
})
- .Case([&](omp::ScopeOp op) {
- checkAllocate(op, result);
- checkReduction(op, result);
- })
+ .Case([&](omp::ScopeOp op) { checkReduction(op, result); })
.Case([&](omp::SingleOp op) {
checkAllocate(op, result);
checkPrivate(op, result);
@@ -1962,16 +1959,44 @@ initPrivateVars(llvm::IRBuilderBase &builder,
return llvm::Error::success();
}
+static LogicalResult
+convertAllocatorVars(Operation &op, ValueRange allocatorVars,
+ llvm::IRBuilderBase &builder,
+ LLVM::ModuleTranslation &moduleTranslation,
+ PrivateVarsInfo &privateVarsInfo) {
+ for (Value allocatorVar : allocatorVars) {
+ if (privateVarsInfo.convertedAllocators.contains(allocatorVar))
+ continue;
+
+ llvm::Value *allocator = moduleTranslation.lookupValue(allocatorVar);
+ if (!allocator)
+ return op.emitError("failed to translate OpenMP allocator operand");
+ if (allocator->getType()->isIntegerTy())
+ allocator = builder.CreateIntToPtr(allocator, builder.getPtrTy());
+ else if (allocator->getType()->isPointerTy())
+ allocator = builder.CreatePointerBitCastOrAddrSpaceCast(
+ allocator, builder.getPtrTy());
+ else
+ return op.emitError(
+ "OpenMP allocator operand must have integer or pointer type");
+
+ privateVarsInfo.convertedAllocators.try_emplace(allocatorVar, allocator);
+ }
+ return success();
+}
+
/// Allocate and initialize delayed private variables. Returns the basic block
/// which comes after all of these allocations. llvm::Value * for each of these
/// private variables are populated in llvmPrivateVars.
template <typename T>
-static llvm::Expected<llvm::BasicBlock *>
-allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
- LLVM::ModuleTranslation &moduleTranslation,
- PrivateVarsInfo &privateVarsInfo,
- const llvm::OpenMPIRBuilder::InsertPointTy &allocaIP,
- llvm::DenseMap<Value, Value> *mappedPrivateVars = nullptr) {
+static llvm::Expected<llvm::BasicBlock *> allocatePrivateVars(
+ T op, llvm::IRBuilderBase &builder,
+ LLVM::ModuleTranslation &moduleTranslation,
+ PrivateVarsInfo &privateVarsInfo,
+ const llvm::OpenMPIRBuilder::InsertPointTy &allocaIP,
+ llvm::DenseMap<Value, Value> *mappedPrivateVars = nullptr,
+ std::optional<llvm::OpenMPIRBuilder::InsertPointTy> allocatorIP =
+ std::nullopt) {
// Allocate private vars
llvm::Instruction *allocaTerminator = allocaIP.getBlock()->getTerminator();
splitBB(llvm::OpenMPIRBuilder::InsertPointTy(allocaIP.getBlock(),
@@ -1979,13 +2004,31 @@ allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
true, allocaTerminator->getStableDebugLoc(),
"omp.region.after_alloca");
- llvm::IRBuilderBase::InsertPointGuard guard(builder);
+ llvm::Instruction *allocatorTerminator = nullptr;
+ llvm::BasicBlock *afterAllocatorAllocations = nullptr;
+ if (allocatorIP) {
+ allocatorTerminator = allocatorIP->getBlock()->getTerminator();
+ afterAllocatorAllocations = splitBB(
+ llvm::OpenMPIRBuilder::InsertPointTy(
+ allocatorIP->getBlock(), allocatorTerminator->getIterator()),
+ true, allocatorTerminator->getStableDebugLoc(),
+ "omp.region.after_allocate");
+ }
+
+ std::optional<llvm::IRBuilderBase::InsertPointGuard> guard;
+ if (!allocatorIP)
+ guard.emplace(builder);
// Update the allocaTerminator since the alloca block was split above.
allocaTerminator = allocaIP.getBlock()->getTerminator();
builder.SetInsertPoint(allocaTerminator);
// The new terminator is an uncondition branch created by the splitBB above.
assert(allocaTerminator->getNumSuccessors() == 1 &&
"This is an unconditional branch created by splitBB");
+ if (allocatorIP) {
+ allocatorTerminator = allocatorIP->getBlock()->getTerminator();
+ assert(allocatorTerminator->getNumSuccessors() == 1 &&
+ "This is an unconditional branch created by splitBB");
+ }
llvm::DataLayout dataLayout = builder.GetInsertBlock()->getDataLayout();
llvm::BasicBlock *afterAllocas = allocaTerminator->getSuccessor(0);
@@ -2002,7 +2045,8 @@ allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
-1);
ValueRange allocatorVars;
DenseI64ArrayAttr allocateAlignments;
- if constexpr (std::is_same_v<T, omp::ParallelOp>) {
+ if constexpr (std::is_same_v<T, omp::ParallelOp> ||
+ std::is_same_v<T, omp::ScopeOp>) {
allocatorVars = op.getAllocatorVars();
allocateAlignments = op.getAllocateAlignmentsAttr();
if (auto privateIndices = op.getAllocatePrivateIndicesAttr())
@@ -2017,14 +2061,16 @@ allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
auto [privDecl, mlirPrivVar, blockArg] = tuple;
llvm::Type *llvmAllocType =
moduleTranslation.convertType(privDecl.getType());
- builder.SetInsertPoint(allocaIP.getBlock()->getTerminator());
llvm::Value *llvmPrivateVar = nullptr;
int64_t allocateIndex = allocateItemForPrivate[privateIndex];
+ builder.SetInsertPoint(allocateIndex >= 0 && allocatorTerminator
+ ? allocatorTerminator
+ : allocaTerminator);
if (allocateIndex >= 0) {
- if (mightUseDeviceSharedMem ||
+ if (ompBuilder->Config.isTargetDevice() ||
op->template getParentOfType<omp::TargetOp>())
return llvm::createStringError(
- "allocate clause on a device parallel region is not supported");
+ "allocate clause in an OpenMP device context is not supported");
if (!llvmAllocType->isSized())
return llvm::createStringError(
"allocate clause private type must have a fixed size");
@@ -2087,7 +2133,7 @@ allocatePrivateVars(T op, llvm::IRBuilderBase &builder,
privateVarsInfo.llvmVars.push_back(llvmPrivateVar);
}
- return afterAllocas;
+ return afterAllocatorAllocations ? afterAllocatorAllocations : afterAllocas;
}
/// This can't always be determined statically, but when we can, it is good to
@@ -2411,10 +2457,14 @@ convertOmpScope(omp::ScopeOp &scopeOp, llvm::IRBuilderBase &builder,
assert(isByRef.size() == scopeOp.getNumReductionVars());
PrivateVarsInfo privateVarsInfo(scopeOp);
+ if (failed(convertAllocatorVars(*scopeOp, scopeOp.getAllocatorVars(), builder,
+ moduleTranslation, privateVarsInfo)))
+ return failure();
SmallVector<omp::DeclareReductionOp> reductionDecls;
collectReductionDecls(scopeOp, reductionDecls);
- InsertPointTy allocaIP = findAllocInsertPoints(builder, moduleTranslation);
+ InsertPointTy privateAllocaIP =
+ findAllocInsertPoints(builder, moduleTranslation);
SmallVector<llvm::Value *> privateReductionVariables(
scopeOp.getNumReductionVars());
@@ -2423,14 +2473,15 @@ convertOmpScope(omp::ScopeOp &scopeOp, llvm::IRBuilderBase &builder,
MutableArrayRef<BlockArgument> reductionArgs =
cast<omp::BlockArgOpenMPOpInterface>(*scopeOp).getReductionBlockArgs();
- // Allocate private vars before the scope body
- llvm::Expected<llvm::BasicBlock *> afterAllocas = allocatePrivateVars(
- scopeOp, builder, moduleTranslation, privateVarsInfo, allocaIP);
- if (failed(handleError(afterAllocas, *scopeOp)))
- return failure();
+ if (scopeOp.getAllocateVars().empty()) {
+ llvm::Expected<llvm::BasicBlock *> afterAllocas = allocatePrivateVars(
+ scopeOp, builder, moduleTranslation, privateVarsInfo, privateAllocaIP);
+ if (failed(handleError(afterAllocas, *scopeOp)))
+ return failure();
+ }
if (failed(allocAndInitializeReductionVars(
- scopeOp, reductionArgs, builder, moduleTranslation, allocaIP,
+ scopeOp, reductionArgs, builder, moduleTranslation, privateAllocaIP,
reductionDecls, privateReductionVariables, reductionVariableMap,
isByRef)))
return failure();
@@ -2438,7 +2489,18 @@ convertOmpScope(omp::ScopeOp &scopeOp, llvm::IRBuilderBase &builder,
auto bodyCB =
[&](InsertPointTy allocaIP, InsertPointTy codeGenIP,
llvm::ArrayRef<llvm::BasicBlock *> deallocBlocks) -> llvm::Error {
- builder.restoreIP(codeGenIP);
+ if (!scopeOp.getAllocateVars().empty()) {
+ // Runtime storage must be allocated on each dynamic Scope entry. Ordinary
+ // private allocas still use the enclosing alloca insertion point.
+ llvm::Expected<llvm::BasicBlock *> afterAllocas = allocatePrivateVars(
+ scopeOp, builder, moduleTranslation, privateVarsInfo, privateAllocaIP,
+ /*mappedPrivateVars=*/nullptr, codeGenIP);
+ if (handleError(afterAllocas, *scopeOp).failed())
+ return llvm::make_error<PreviouslyReportedError>();
+ builder.SetInsertPoint(afterAllocas.get()->getTerminator());
+ } else {
+ builder.restoreIP(codeGenIP);
+ }
if (handleError(
initPrivateVars(builder, moduleTranslation, privateVarsInfo),
@@ -2478,7 +2540,7 @@ convertOmpScope(omp::ScopeOp &scopeOp, llvm::IRBuilderBase &builder,
// Process the reductions if required.
return createReductionsAndCleanup(
- scopeOp, builder, moduleTranslation, allocaIP, reductionDecls,
+ scopeOp, builder, moduleTranslation, privateAllocaIP, reductionDecls,
privateReductionVariables, isByRef, scopeOp.getNowait(),
/*isTeamsReduction=*/false);
}
@@ -4899,24 +4961,9 @@ convertOmpParallel(omp::ParallelOp opInst, llvm::IRBuilderBase &builder,
return failure();
PrivateVarsInfo privateVarsInfo(opInst);
- for (Value allocatorVar : opInst.getAllocatorVars()) {
- if (privateVarsInfo.convertedAllocators.contains(allocatorVar))
- continue;
-
- llvm::Value *allocator = moduleTranslation.lookupValue(allocatorVar);
- if (!allocator)
- return opInst.emitError("failed to translate OpenMP allocator operand");
- if (allocator->getType()->isIntegerTy())
- allocator = builder.CreateIntToPtr(allocator, builder.getPtrTy());
- else if (allocator->getType()->isPointerTy())
- allocator = builder.CreatePointerBitCastOrAddrSpaceCast(
- allocator, builder.getPtrTy());
- else
- return opInst.emitError(
- "OpenMP allocator operand must have integer or pointer type");
-
- privateVarsInfo.convertedAllocators.try_emplace(allocatorVar, allocator);
- }
+ if (failed(convertAllocatorVars(*opInst, opInst.getAllocatorVars(), builder,
+ moduleTranslation, privateVarsInfo)))
+ return failure();
// Collect reduction declarations
SmallVector<omp::DeclareReductionOp> reductionDecls;
diff --git a/mlir/test/Dialect/OpenMP/invalid.mlir b/mlir/test/Dialect/OpenMP/invalid.mlir
index 73f63e13f90f6..c3fe4f60e4166 100644
--- a/mlir/test/Dialect/OpenMP/invalid.mlir
+++ b/mlir/test/Dialect/OpenMP/invalid.mlir
@@ -3530,6 +3530,79 @@ func.func @omp_parallel_allocate_type_mismatch(
// -----
+omp.private {type = private} @scope_allocate_private : i32
+
+func.func @omp_scope_allocate_missing_map(%allocator : i64, %var : !llvm.ptr) {
+ // expected-error @below {{expected an allocate private index for each allocate variable}}
+ omp.scope allocate(%allocator : i64 -> %var : !llvm.ptr)
+ private(@scope_allocate_private %var -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
+omp.private {type = private} @scope_allocate_private : i32
+
+func.func @omp_scope_allocate_map_range(%allocator : i64, %var : !llvm.ptr) {
+ // expected-error @below {{allocate private index is out of range}}
+ omp.scope allocate(%allocator : i64 -> %var : !llvm.ptr) allocate_private_indices([1])
+ private(@scope_allocate_private %var -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
+omp.private {type = private} @scope_x_private : i32
+omp.private {type = private} @scope_y_private : i32
+
+func.func @omp_scope_allocate_map_duplicate(
+ %allocator : i64, %x : !llvm.ptr, %y : !llvm.ptr) {
+ // expected-error @below {{allocate private index refers to a private variable more than once}}
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr,
+ %allocator : i64 -> %y : !llvm.ptr) allocate_private_indices([0, 0])
+ private(@scope_x_private %x -> %x_private,
+ @scope_y_private %y -> %y_private : !llvm.ptr, !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
+omp.private {type = private} @scope_allocate_private : i32
+
+func.func @omp_scope_allocate_type_mismatch(
+ %allocator : i64, %allocate_var : i64, %private_var : !llvm.ptr) {
+ // expected-error @below {{type mismatch between allocate variable and private variable at index 0}}
+ omp.scope allocate(%allocator : i64 -> %allocate_var : i64) allocate_private_indices([0])
+ private(@scope_allocate_private %private_var -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
+omp.private {type = private} @scope_x_private : i32
+omp.private {type = private} @scope_y_private : i32
+
+func.func @omp_scope_allocate_wrong_private_slot(
+ %allocator : i64, %x : !llvm.ptr, %y : !llvm.ptr) {
+ // expected-error @below {{allocate variable does not match private variable at index 1}}
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([1])
+ private(@scope_x_private %x -> %x_private,
+ @scope_y_private %y -> %y_private : !llvm.ptr, !llvm.ptr) {
+ omp.terminator
+ }
+ return
+}
+
+// -----
+
func.func @omp_distribute_nested_wrapper(%lb: index, %ub: index, %step: index) -> () {
// expected-error @below {{an 'omp.wsloop' nested wrapper is only allowed when a composite 'omp.parallel' is the direct parent}}
omp.distribute {
diff --git a/mlir/test/Dialect/OpenMP/ops.mlir b/mlir/test/Dialect/OpenMP/ops.mlir
index 49646e3c64ddf..9371d3599b72a 100644
--- a/mlir/test/Dialect/OpenMP/ops.mlir
+++ b/mlir/test/Dialect/OpenMP/ops.mlir
@@ -157,6 +157,7 @@ func.func @omp_parallel(%data_var : memref<i32>, %if_cond : i1, %num_threads : i
}
omp.private {type = private} @parallel_allocate_private : memref<i32>
+omp.private {type = private} @parallel_allocate_associated_private : memref<i32>
func.func @omp_parallel_pretty(%data_var : memref<i32>, %if_cond : i1, %num_threads : i32, %allocator : si32) -> () {
// CHECK: omp.parallel
@@ -201,6 +202,16 @@ func.func @omp_parallel_pretty(%data_var : memref<i32>, %if_cond : i1, %num_thre
omp.terminator
}
+ // CHECK: omp.parallel allocate(
+ // CHECK-SAME: allocate_private_indices([0, 1])
+ // CHECK-SAME: private(@parallel_allocate_private %[[STORAGE:.*]] -> {{.*}}, @parallel_allocate_associated_private %[[STORAGE]] -> {{.*}} : memref<i32>, memref<i32>)
+ omp.parallel allocate(%allocator : si32 -> %data_var : memref<i32>,
+ %allocator : si32 -> %data_var : memref<i32>) allocate_private_indices([0, 1])
+ private(@parallel_allocate_private %data_var -> %private.x,
+ @parallel_allocate_associated_private %data_var -> %private.y : memref<i32>, memref<i32>) {
+ omp.terminator
+ }
+
// CHECK: omp.parallel
// CHECK-NEXT: omp.parallel if(%{{.*}})
omp.parallel {
@@ -218,6 +229,32 @@ func.func @omp_parallel_pretty(%data_var : memref<i32>, %if_cond : i1, %num_thre
return
}
+omp.private {type = private} @scope_allocate_private : memref<i32>
+omp.private {type = private} @scope_allocate_associated_private : memref<i32>
+
+// CHECK-LABEL: omp_scope_pretty
+func.func @omp_scope_pretty(%data_var : memref<i32>, %allocator : si32) -> () {
+ // CHECK: omp.scope allocate(
+ // CHECK-SAME: allocate_alignments([64]) allocate_private_indices([0])
+ // CHECK-SAME: private(
+ omp.scope allocate(%allocator : si32 -> %data_var : memref<i32>) allocate_alignments([64]) allocate_private_indices([0])
+ private(@scope_allocate_private %data_var -> %private : memref<i32>) {
+ omp.terminator
+ }
+
+ // CHECK: omp.scope allocate(
+ // CHECK-SAME: allocate_private_indices([0, 1])
+ // CHECK-SAME: private(@scope_allocate_private %[[STORAGE:.*]] -> {{.*}}, @scope_allocate_associated_private %[[STORAGE]] -> {{.*}} : memref<i32>, memref<i32>)
+ omp.scope allocate(%allocator : si32 -> %data_var : memref<i32>,
+ %allocator : si32 -> %data_var : memref<i32>) allocate_private_indices([0, 1])
+ private(@scope_allocate_private %data_var -> %private.x,
+ @scope_allocate_associated_private %data_var -> %private.y : memref<i32>, memref<i32>) {
+ omp.terminator
+ }
+
+ return
+}
+
// CHECK-LABEL: omp_loop_nest
func.func @omp_loop_nest(%lb : index, %ub : index, %step : index) -> () {
omp.wsloop {
diff --git a/mlir/test/Target/LLVMIR/openmp-allocate-clause.mlir b/mlir/test/Target/LLVMIR/openmp-allocate-clause.mlir
index 8ae2ce9f4a183..baf2c58cd3801 100644
--- a/mlir/test/Target/LLVMIR/openmp-allocate-clause.mlir
+++ b/mlir/test/Target/LLVMIR/openmp-allocate-clause.mlir
@@ -6,6 +6,7 @@
// RUN: not mlir-translate -mlir-to-llvmir %t/i386-overflow.mlir 2>&1 | FileCheck %s --check-prefix=I386-OVERFLOW
// RUN: not mlir-translate -mlir-to-llvmir %t/i386-alignment-overflow.mlir 2>&1 | FileCheck %s --check-prefix=I386-ALIGNMENT-OVERFLOW
// RUN: not mlir-translate -mlir-to-llvmir %t/device.mlir 2>&1 | FileCheck %s --check-prefix=DEVICE
+// RUN: not mlir-translate -mlir-to-llvmir %t/declare-target-device.mlir 2>&1 | FileCheck %s --check-prefix=DECLARE-TARGET-DEVICE
//--- valid.mlir
@@ -31,6 +32,35 @@ llvm.func @allocator_unaligned(%x: !llvm.ptr) {
// CHECK: %[[UNALIGNED:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
// CHECK: call void @__kmpc_free({{.*}}, ptr %[[UNALIGNED]], ptr null)
+llvm.func @allocator_equivalent_storage(%storage: !llvm.ptr) {
+ %null = llvm.mlir.constant(0 : i64) : i64
+ omp.parallel allocate(%null : i64 -> %storage : !llvm.ptr,
+ %null : i64 -> %storage : !llvm.ptr) allocate_private_indices([0, 1])
+ private(@x.firstprivate %storage -> %x.private,
+ @y.private %storage -> %y.private : !llvm.ptr, !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ %two = llvm.mlir.constant(2 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ llvm.store %two, %y.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// CHECK-LABEL: define internal void @allocator_equivalent_storage..omp_par
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[EQUIVALENT_X:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[EQUIVALENT_Y:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 1, ptr %[[EQUIVALENT_X]], align 4
+// CHECK: store i32 2, ptr %[[EQUIVALENT_Y]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[EQUIVALENT_Y]], ptr null)
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[EQUIVALENT_X]], ptr null)
+// CHECK-NOT: call void @__kmpc_free
+
llvm.func @allocator_dynamic(%x: !llvm.ptr, %y: !llvm.ptr, %allocator: i64) {
omp.parallel allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_alignments([64]) allocate_private_indices([0])
private(@x.firstprivate %x -> %x.private,
@@ -97,12 +127,18 @@ llvm.func @allocator_reverse_free(%x: !llvm.ptr, %y: !llvm.ptr,
// CHECK-LABEL: define internal void @allocator_reverse_free..omp_par
// CHECK: %[[CAPTURED_X:.*]] = load ptr, ptr %{{.*}}, align 8
// CHECK: %[[CAPTURED_Y:.*]] = load ptr, ptr %{{.*}}, align 8
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
// CHECK: %[[X_ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 64, i64 4, ptr %[[CAPTURED_X]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
// CHECK: %[[Y_ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 128, i64 4, ptr %[[CAPTURED_Y]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
// CHECK: store i32 1, ptr %[[X_ALLOC]], align 4
// CHECK: store i32 1, ptr %[[Y_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
// CHECK: call void @private_dealloc(ptr %[[X_ALLOC]])
+// CHECK-NOT: call void @__kmpc_free
// CHECK: call void @__kmpc_free({{.*}}, ptr %[[Y_ALLOC]], ptr %[[CAPTURED_Y]])
+// CHECK-NOT: call void @__kmpc_free
// CHECK: call void @__kmpc_free({{.*}}, ptr %[[X_ALLOC]], ptr %[[CAPTURED_X]])
// CHECK-NOT: call void @__kmpc_free
@@ -304,5 +340,30 @@ llvm.func @allocator_device() {
llvm.return
}
-// DEVICE: allocate clause on a device parallel region is not supported
+// DEVICE: allocate clause in an OpenMP device context is not supported
// DEVICE: LLVM Translation failed for operation: omp.parallel
+
+//--- declare-target-device.mlir
+
+module attributes {omp.is_target_device = true} {
+ omp.private {type = private} @device.private : i32
+
+ llvm.func @parallel_allocator_declare_target()
+ attributes {omp.declare_target = #omp.declaretarget<device_type = any, capture_clause = to>} {
+ %one = llvm.mlir.constant(1 : i64) : i64
+ %allocator = llvm.mlir.constant(0 : i64) : i64
+ %x = llvm.alloca %one x i32 : (i64) -> !llvm.ptr
+ omp.parallel allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@device.private %x -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ llvm.return
+ }
+}
+
+// DECLARE-TARGET-DEVICE-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// DECLARE-TARGET-DEVICE-NOT: call void @__kmpc_free
+// DECLARE-TARGET-DEVICE: allocate clause in an OpenMP device context is not supported
+// DECLARE-TARGET-DEVICE: LLVM Translation failed for operation: omp.parallel
+// DECLARE-TARGET-DEVICE-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// DECLARE-TARGET-DEVICE-NOT: call void @__kmpc_free
diff --git a/mlir/test/Target/LLVMIR/openmp-scope-allocate-clause.mlir b/mlir/test/Target/LLVMIR/openmp-scope-allocate-clause.mlir
new file mode 100644
index 0000000000000..a430fc69df4a8
--- /dev/null
+++ b/mlir/test/Target/LLVMIR/openmp-scope-allocate-clause.mlir
@@ -0,0 +1,303 @@
+// RUN: split-file %s %t
+// RUN: mlir-translate -mlir-to-llvmir -split-input-file %t/valid.mlir | FileCheck %s
+// RUN: not mlir-translate -mlir-to-llvmir %t/device.mlir 2>&1 | FileCheck %s --check-prefix=DEVICE
+// RUN: not mlir-translate -mlir-to-llvmir %t/declare-target-device.mlir 2>&1 | FileCheck %s --check-prefix=DECLARE-TARGET-DEVICE
+
+//--- valid.mlir
+
+omp.private {type = private} @x.private : i32
+omp.private {type = private} @y.private : i32
+
+llvm.func @scope_allocator_unaligned(%x: !llvm.ptr, %y: !llvm.ptr) {
+ %null = llvm.mlir.constant(0 : i64) : i64
+ omp.scope allocate(%null : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@x.private %x -> %x.private,
+ @y.private %y -> %y.private : !llvm.ptr, !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ llvm.store %one, %y.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_unaligned
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[Y_ALLOCA:.*]] = alloca i32, align 4
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: %[[UNALIGNED:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: store i32 1, ptr %[[UNALIGNED]], align 4
+// CHECK: store i32 1, ptr %[[Y_ALLOCA]], align 4
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[UNALIGNED]], ptr null)
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: ret void
+
+llvm.func @scope_allocator_equivalent_storage(%storage: !llvm.ptr) {
+ %null = llvm.mlir.constant(0 : i64) : i64
+ omp.scope allocate(%null : i64 -> %storage : !llvm.ptr,
+ %null : i64 -> %storage : !llvm.ptr) allocate_private_indices([0, 1])
+ private(@x.private %storage -> %x.private,
+ @y.private %storage -> %y.private : !llvm.ptr, !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ %two = llvm.mlir.constant(2 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ llvm.store %two, %y.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_equivalent_storage
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[EQUIVALENT_X:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[EQUIVALENT_Y:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr null)
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 1, ptr %[[EQUIVALENT_X]], align 4
+// CHECK: store i32 2, ptr %[[EQUIVALENT_Y]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[EQUIVALENT_Y]], ptr null)
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[EQUIVALENT_X]], ptr null)
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: ret void
+
+// -----
+
+omp.private {type = private} @constant.private : i32
+
+llvm.func @scope_allocator_constant(%x: !llvm.ptr) {
+ %allocator = llvm.mlir.constant(3 : i64) : i64
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@constant.private %x -> %x.private : !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_constant
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[CONSTANT_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr inttoptr (i64 3 to ptr))
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 1, ptr %[[CONSTANT_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[CONSTANT_ALLOC]], ptr inttoptr (i64 3 to ptr))
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: ret void
+
+// -----
+
+llvm.func @private_dealloc(!llvm.ptr)
+
+omp.private {type = firstprivate} @x.private : i32 copy {
+^bb0(%original: !llvm.ptr, %private: !llvm.ptr):
+ %value = llvm.load %original : !llvm.ptr -> i32
+ llvm.store %value, %private : i32, !llvm.ptr
+ omp.yield(%private : !llvm.ptr)
+} dealloc {
+^bb0(%private: !llvm.ptr):
+ llvm.call @private_dealloc(%private) : (!llvm.ptr) -> ()
+ omp.yield
+}
+
+llvm.func @scope_allocator_firstprivate_aligned(%x: !llvm.ptr, %allocator: i64) {
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_alignments([64]) allocate_private_indices([0])
+ private(@x.private %x -> %x.private : !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_firstprivate_aligned
+// CHECK: %[[ALLOCATOR:.*]] = inttoptr i64 %{{.*}} to ptr
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 64, i64 4, ptr %[[ALLOCATOR]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[ORIGINAL:.*]] = load i32, ptr %{{.*}}, align 4
+// CHECK: store i32 %[[ORIGINAL]], ptr %[[ALLOC]], align 4
+// CHECK: store i32 1, ptr %[[ALLOC]], align 4
+// CHECK: call void @private_dealloc(ptr %[[ALLOC]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[ALLOC]], ptr %[[ALLOCATOR]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: ret void
+
+// -----
+
+omp.private {type = private} @x.private : i32
+omp.private {type = private} @y.private : i32
+
+llvm.func @scope_allocator_order(%x: !llvm.ptr, %y: !llvm.ptr,
+ %allocator.x: i64, %allocator.y: i64) {
+ omp.scope allocate(%allocator.y : i64 -> %y : !llvm.ptr,
+ %allocator.x : i64 -> %x : !llvm.ptr) allocate_alignments([128, 64]) allocate_private_indices([1, 0])
+ private(@x.private %x -> %x.private,
+ @y.private %y -> %y.private : !llvm.ptr, !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ llvm.store %one, %y.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.return
+}
+
+// Allocation-list order (y, x)
diff ers from private-list order (x, y); the
+// allocate_private_indices mapping must still route each allocation to the
+// correct private slot, and cleanup must free exactly once per allocation.
+// CHECK-LABEL: define void @scope_allocator_order
+// CHECK: %[[ALLOCATOR_Y:.*]] = inttoptr i64 %{{.*}} to ptr
+// CHECK: %[[ALLOCATOR_X:.*]] = inttoptr i64 %{{.*}} to ptr
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[X_ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 64, i64 4, ptr %[[ALLOCATOR_X]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: %[[Y_ALLOC:.*]] = call ptr @__kmpc_aligned_alloc(i32 %{{.*}}, i64 128, i64 4, ptr %[[ALLOCATOR_Y]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 1, ptr %[[X_ALLOC]], align 4
+// CHECK: store i32 1, ptr %[[Y_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[Y_ALLOC]], ptr %[[ALLOCATOR_Y]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[X_ALLOC]], ptr %[[ALLOCATOR_X]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: ret void
+
+// -----
+
+omp.private {type = private} @conditional.private : i32
+
+llvm.func @scope_allocator_conditional(%x: !llvm.ptr, %allocator: i64,
+ %condition: i1) {
+ llvm.cond_br %condition, ^scope, ^exit
+^scope:
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@conditional.private %x -> %x.private : !llvm.ptr) {
+ %one = llvm.mlir.constant(1 : i32) : i32
+ llvm.store %one, %x.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ llvm.br ^exit
+^exit:
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_conditional
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: br i1 {{.*}}, label %[[SCOPE:.*]], label %[[EXIT:.*]]
+// CHECK: [[SCOPE]]:
+// CHECK: %[[CONDITIONAL_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr %{{.*}})
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 1, ptr %[[CONDITIONAL_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[CONDITIONAL_ALLOC]], ptr %{{.*}})
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: br label %[[EXIT]]
+// CHECK: [[EXIT]]:
+// CHECK: ret void
+
+// -----
+
+omp.private {type = private} @loop.private : i32
+
+llvm.func @scope_allocator_loop(%x: !llvm.ptr, %allocator.base: i64, %n: i64) {
+ %zero = llvm.mlir.constant(0 : i64) : i64
+ %one = llvm.mlir.constant(1 : i64) : i64
+ llvm.br ^loop(%zero : i64)
+^loop(%i: i64):
+ %condition = llvm.icmp "slt" %i, %n : i64
+ llvm.cond_br %condition, ^body, ^exit
+^body:
+ %allocator = llvm.add %allocator.base, %i : i64
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@loop.private %x -> %x.private : !llvm.ptr) {
+ %value = llvm.trunc %i : i64 to i32
+ llvm.store %value, %x.private : i32, !llvm.ptr
+ omp.terminator
+ }
+ %next = llvm.add %i, %one : i64
+ llvm.br ^loop(%next : i64)
+^exit:
+ llvm.return
+}
+
+// CHECK-LABEL: define void @scope_allocator_loop
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: omp.region.after_alloca:
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: br label %[[LOOP:[0-9]+]]
+// CHECK: [[LOOP]]:
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: br i1 {{.*}}, label %[[BODY:[0-9]+]], label %[[LOOP_EXIT:[0-9]+]]
+// CHECK: [[BODY]]:
+// CHECK: %[[DYNAMIC_ALLOCATOR:.*]] = add i64 %{{.*}}, %{{.*}}
+// CHECK: %[[ALLOCATOR_HANDLE:.*]] = inttoptr i64 %[[DYNAMIC_ALLOCATOR]] to ptr
+// CHECK: %[[LOOP_ALLOC:.*]] = call ptr @__kmpc_alloc({{.*}}, i64 4, ptr %[[ALLOCATOR_HANDLE]])
+// CHECK-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// CHECK: store i32 {{.*}}, ptr %[[LOOP_ALLOC]], align 4
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: call void @__kmpc_free({{.*}}, ptr %[[LOOP_ALLOC]], ptr %[[ALLOCATOR_HANDLE]])
+// CHECK-NOT: call void @__kmpc_free
+// CHECK: br label %[[LOOP]]
+// CHECK: [[LOOP_EXIT]]:
+// CHECK: ret void
+
+//--- device.mlir
+
+omp.private {type = private} @device.private : i32
+
+llvm.func @scope_allocator_device() {
+ omp.target kernel_type(generic) {
+ %allocator = llvm.mlir.constant(0 : i64) : i64
+ %x = llvm.alloca %allocator x i32 : (i64) -> !llvm.ptr
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_alignments([64]) allocate_private_indices([0])
+ private(@device.private %x -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ omp.terminator
+ }
+ llvm.return
+}
+
+// DEVICE-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// DEVICE: allocate clause in an OpenMP device context is not supported
+// DEVICE: LLVM Translation failed for operation: omp.scope
+// DEVICE-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+
+//--- declare-target-device.mlir
+
+module attributes {omp.is_target_device = true} {
+ omp.private {type = private} @device.private : i32
+
+ llvm.func @scope_allocator_declare_target()
+ attributes {
+ omp.declare_target =
+ #omp.declaretarget<device_type = any, capture_clause = to>
+ } {
+ %one = llvm.mlir.constant(1 : i64) : i64
+ %allocator = llvm.mlir.constant(0 : i64) : i64
+ %x = llvm.alloca %one x i32 : (i64) -> !llvm.ptr
+ omp.parallel {
+ omp.scope allocate(%allocator : i64 -> %x : !llvm.ptr) allocate_private_indices([0])
+ private(@device.private %x -> %private : !llvm.ptr) {
+ omp.terminator
+ }
+ omp.terminator
+ }
+ llvm.return
+ }
+}
+
+// DECLARE-TARGET-DEVICE-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// DECLARE-TARGET-DEVICE-NOT: call void @__kmpc_free
+// DECLARE-TARGET-DEVICE: allocate clause in an OpenMP device context is not supported
+// DECLARE-TARGET-DEVICE: LLVM Translation failed for operation: omp.scope
+// DECLARE-TARGET-DEVICE-NOT: call ptr @__kmpc_{{(aligned_)?}}alloc
+// DECLARE-TARGET-DEVICE-NOT: call void @__kmpc_free
diff --git a/mlir/test/Target/LLVMIR/openmp-todo.mlir b/mlir/test/Target/LLVMIR/openmp-todo.mlir
index 43063cdf1c82d..ae4f539504a08 100644
--- a/mlir/test/Target/LLVMIR/openmp-todo.mlir
+++ b/mlir/test/Target/LLVMIR/openmp-todo.mlir
@@ -77,17 +77,6 @@ llvm.func @sections_allocate(%x : !llvm.ptr) {
// -----
-llvm.func @scope_allocate(%x : !llvm.ptr) {
- // expected-error at below {{not yet implemented: Unhandled clause allocate in omp.scope operation}}
- // expected-error at below {{LLVM Translation failed for operation: omp.scope}}
- omp.scope allocate(%x : !llvm.ptr -> %x : !llvm.ptr) {
- omp.terminator
- }
- llvm.return
-}
-
-// -----
-
omp.private {type = private} @x.privatizer : i32 init {
^bb0(%mold: !llvm.ptr, %private: !llvm.ptr):
%c0 = llvm.mlir.constant(0 : i32) : i32
More information about the flang-commits
mailing list