[llvm-branch-commits] [flang] [mlir] [Flang][OpenMP] Improve implicit declare_target propagation (PR #214184)
Sergio Afonso via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Wed Aug 5 03:19:23 PDT 2026
https://github.com/skatrak created https://github.com/llvm/llvm-project/pull/214184
After starting to run the `MarkDeclareTarget` pass later in the pipeline, some limitations of its original implementation started to be hit; specifically, some calls being missed could result in an overly restrictive marking that would cause the `HostOpFiltering` pass to remove reachable device code.
This patch aims to address these problems by making the following changes:
- It makes sure to mark functions in every `RecipeInterface` op pointed to by OpenMP operations.
- It recursively propagates and combines declare_target information from target regions and explicitly set declare_target functions to unmarked functions, but it never modifies explicitly marked functions.
- External and public functions can now only be marked with `device_type(any)`. Before, marking them as `nohost` or `host` was possible, but without the ability to see all users we can't give such guarantees.
- There was a bug in the handling of declare_target on interface subprograms that improvements to the pass made visible. Instead of adding the attribute to the MLIR function associated to the interface subprogram, it wasn't added at all, making it susceptible to being modified by the pass according to uses. This is fixed.
Fixes #213367.
>From 436ddceaf6cc388a2c5b02ecb4399df02fceaad7 Mon Sep 17 00:00:00 2001
From: Sergio Afonso <Sergio.AfonsoFumero at amd.com>
Date: Wed, 5 Aug 2026 11:01:19 +0100
Subject: [PATCH] [Flang][OpenMP] Improve implicit declare_target propagation
After starting to run the `MarkDeclareTarget` pass later in the
pipeline, some limitations of its original implementation started to be
hit; specifically, some calls being missed could result in an overly
restrictive marking that would cause the `HostOpFiltering` pass to
remove reachable device code.
This patch aims to address these problems by making the following
changes:
- It makes sure to mark functions in every `RecipeInterface` op pointed
to by OpenMP operations.
- It recursively propagates and combines declare_target information from
target regions and explicitly set declare_target functions to unmarked
functions, but it never modifies explicitly marked functions.
- External and public functions can now only be marked with
`device_type(any)`. Before, marking them as `nohost` or `host` was
possible, but without the ability to see all users we can't give such
guarantees.
- There was a bug in the handling of declare_target on interface
subprograms that improvements to the pass made visible. Instead of
adding the attribute to the MLIR function associated to the interface
subprogram, it wasn't added at all, making it susceptible to being
modified by the pass according to uses. This is fixed.
Fixes #213367.
---
flang/lib/Lower/OpenMP/OpenMP.cpp | 36 +-
.../OpenMP/function-filtering-2.f90 | 90 ++---
...are-target-deferred-marking-reductions.f90 | 6 +-
.../declare-target-deferred-marking.f90 | 2 +-
...arget-implicit-func-and-subr-cap-enter.f90 | 251 ++++++------
...lare-target-implicit-func-and-subr-cap.f90 | 293 +++++++-------
.../declare-target-implicit-tarop-cap.f90 | 171 +++++----
.../declare-target-named-main-interface.f90 | 4 +-
.../mlir/Dialect/OpenMP/Transforms/Passes.td | 3 +
.../OpenMP/Transforms/MarkDeclareTarget.cpp | 343 ++++++++++++-----
.../Dialect/OpenMP/mark-declare-target.mlir | 363 ++++++++++++++++++
11 files changed, 1067 insertions(+), 495 deletions(-)
create mode 100644 mlir/test/Dialect/OpenMP/mark-declare-target.mlir
diff --git a/flang/lib/Lower/OpenMP/OpenMP.cpp b/flang/lib/Lower/OpenMP/OpenMP.cpp
index ae1eb897c9348..f6711bff65a2e 100644
--- a/flang/lib/Lower/OpenMP/OpenMP.cpp
+++ b/flang/lib/Lower/OpenMP/OpenMP.cpp
@@ -1433,16 +1433,38 @@ static void getDeclareTargetInfo(
} else {
List<Clause> clauses = makeClauses(construct.v.Clauses(), semaCtx);
if (clauses.empty()) {
+ // Case: implicit capture of the enclosing function/subroutine.
Fortran::lower::pft::FunctionLikeUnit *owningProc =
eval.getOwningProcedure();
- // Main programs are never device routines. Skip them so that a bare
- // '!$omp declare target' inside an interface body that lives in a named
- // main program does not incorrectly mark _QQmain as a device function.
- if (owningProc && !owningProc->isMainProgram()) {
- // Case: declare target, implicit capture of enclosing
- // function/subroutine.
+ bool owningProcNotMainProgram =
+ owningProc && !owningProc->isMainProgram();
+
+ const semantics::Symbol *owningSym =
+ owningProcNotMainProgram
+ ? &owningProc->getSubprogramSymbol()
+ : (owningProc ? owningProc->getMainProgramSymbol() : nullptr);
+
+ // A bare '!$omp declare target' may appear in the specification part of
+ // an interface body. In that case the PFT records the directive as an
+ // evaluation of the enclosing program unit rather than of the interface
+ // body's subprogram, so eval.getOwningProcedure() points at the main
+ // program. Detect this by comparing the program unit lexically containing
+ // the directive with the procedure currently being lowered; when they
+ // differ, the directive belongs to the interface-body subprogram, which
+ // is the symbol we must capture.
+ const semantics::Scope &progUnitScope =
+ semantics::GetProgramUnitContaining(
+ semaCtx.FindScope(construct.v.source));
+ const semantics::Symbol *lexicalSym = progUnitScope.symbol();
+
+ if (lexicalSym && lexicalSym != owningSym) {
+ // Interface subprogram capture.
symbolAndClause.emplace_back(mlir::omp::DeclareTargetCaptureClause::to,
- owningProc->getSubprogramSymbol());
+ *lexicalSym);
+ } else if (owningProcNotMainProgram) {
+ // Main programs are never device routines, so skip those here.
+ symbolAndClause.emplace_back(mlir::omp::DeclareTargetCaptureClause::to,
+ *owningSym);
}
}
diff --git a/flang/test/Integration/OpenMP/function-filtering-2.f90 b/flang/test/Integration/OpenMP/function-filtering-2.f90
index eda3015811e73..ee0c68adc78b3 100644
--- a/flang/test/Integration/OpenMP/function-filtering-2.f90
+++ b/flang/test/Integration/OpenMP/function-filtering-2.f90
@@ -13,49 +13,51 @@
! RUN: bbc -fopenmp -fopenmp-version=52 -emit-hlfir %s -o - | tco -test-gen | FileCheck --check-prefixes=MLIR-ALL,MLIR-HOST %s
! RUN: %if amdgpu-registered-target %{ bbc -target amdgcn-amd-amdhsa -fopenmp -fopenmp-version=52 -fopenmp-is-target-device -emit-hlfir %s -o - | tco -test-gen | FileCheck --check-prefixes=MLIR-ALL,MLIR-DEVICE %s %}
-! MLIR-ALL: llvm.func @{{.*}}implicit_invocation() attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-! MLIR-ALL: llvm.return
-! LLVM-ALL: define {{.*}} @{{.*}}implicit_invocation{{.*}}(
-subroutine implicit_invocation()
-end subroutine implicit_invocation
-
-! MLIR-ALL: llvm.func @{{.*}}declaretarget() attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-! MLIR-ALL: llvm.return
-! LLVM-ALL: define {{.*}} @{{.*}}declaretarget{{.*}}(
-subroutine declaretarget()
-!$omp declare target to(declaretarget) device_type(nohost)
- call implicit_invocation()
-end subroutine declaretarget
-
-! MLIR-ALL: llvm.func @{{.*}}declaretarget_enter() attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
-! MLIR-ALL: llvm.return
-! LLVM-ALL: define {{.*}} @{{.*}}declaretarget_enter{{.*}}(
-subroutine declaretarget_enter()
-!$omp declare target enter(declaretarget_enter) device_type(nohost)
- call implicit_invocation()
-end subroutine declaretarget_enter
-
-! MLIR-ALL: llvm.func @{{.*}}no_declaretarget() attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-! MLIR-ALL: llvm.return
-! LLVM-ALL: define {{.*}} @{{.*}}no_declaretarget{{.*}}(
-subroutine no_declaretarget()
-end subroutine no_declaretarget
-
-! MLIR-ALL: llvm.func @{{.+}}main(
-! MLIR-ALL: omp.target
-! MLIR-ALL: llvm.return
-
-! MLIR-HOST: llvm.func @main(
-! MLIR-HOST: llvm.return
-! MLIR-DEVICE-NOT: llvm.func @main(
-
-! LLVM-HOST: define {{.*}} @{{.*}}main{{.*}}(
-! LLVM-HOST: {{.*}} @{{.*}}__omp_offloading{{.*}}main_{{.*}}(
-! LLVM-DEVICE-NOT: {{.*}} @{{.*}}main{{.*}}(
-! LLVM-DEVICE: define {{.*}} @{{.*}}__omp_offloading{{.*}}main_{{.*}}(
program main
-!$omp target
- call declaretarget()
- call no_declaretarget()
-!$omp end target
+ ! MLIR-ALL: llvm.func @{{.+}}main(
+ ! MLIR-ALL: omp.target
+ ! MLIR-ALL: llvm.return
+ !$omp target
+ call declaretarget()
+ call declaretarget_enter()
+ call no_declaretarget()
+ !$omp end target
+
+ contains
+ ! MLIR-ALL: llvm.func{{.*}} @{{.*}}implicit_invocation() attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ ! MLIR-ALL: llvm.return
+ ! LLVM-ALL: define {{.*}} @{{.*}}implicit_invocation{{.*}}(
+ subroutine implicit_invocation()
+ end subroutine implicit_invocation
+
+ ! MLIR-ALL: llvm.func{{.*}} @{{.*}}declaretarget() attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ ! MLIR-ALL: llvm.return
+ ! LLVM-ALL: define {{.*}} @{{.*}}declaretarget{{.*}}(
+ subroutine declaretarget()
+ !$omp declare target to(declaretarget) device_type(nohost)
+ call implicit_invocation()
+ end subroutine declaretarget
+
+ ! MLIR-ALL: llvm.func{{.*}} @{{.*}}declaretarget_enter() attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
+ ! MLIR-ALL: llvm.return
+ ! LLVM-ALL: define {{.*}} @{{.*}}declaretarget_enter{{.*}}(
+ subroutine declaretarget_enter()
+ !$omp declare target enter(declaretarget_enter) device_type(nohost)
+ call implicit_invocation()
+ end subroutine declaretarget_enter
+
+ ! MLIR-ALL: llvm.func{{.*}} @{{.*}}no_declaretarget() attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ ! MLIR-ALL: llvm.return
+ ! LLVM-ALL: define {{.*}} @{{.*}}no_declaretarget{{.*}}(
+ subroutine no_declaretarget()
+ end subroutine no_declaretarget
+
+ ! MLIR-HOST: llvm.func{{.*}} @main(
+ ! MLIR-DEVICE-NOT: llvm.func{{.*}} @main(
+ ! MLIR-HOST: llvm.return
+
+ ! LLVM-HOST: define {{.*}} @{{.*}}main{{.*}}(
+ ! LLVM-HOST: {{.*}} @{{.*}}__omp_offloading{{.*}}main_{{.*}}(
+ ! LLVM-DEVICE-NOT: {{.*}} @{{.*}}main{{.*}}(
+ ! LLVM-DEVICE: define {{.*}} @{{.*}}__omp_offloading{{.*}}main_{{.*}}(
end program main
diff --git a/flang/test/Lower/OpenMP/declare-target-deferred-marking-reductions.f90 b/flang/test/Lower/OpenMP/declare-target-deferred-marking-reductions.f90
index b245f5fdc560b..a95561389364c 100644
--- a/flang/test/Lower/OpenMP/declare-target-deferred-marking-reductions.f90
+++ b/flang/test/Lower/OpenMP/declare-target-deferred-marking-reductions.f90
@@ -31,6 +31,6 @@ end function mycombine
end program main
!CHECK: llvm.func @myinit(!llvm.ptr, !llvm.ptr)
-!CHECK-SAME: {{.*}}, omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}
-!CHECK: llvm.func @mycombine(!llvm.ptr, !llvm.ptr)
-!CHECK-SAME: {{.*}}, omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}
+!CHECK-SAME: {{.*}}, omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}
+!CHECK-LABEL: llvm.func @mycombine(!llvm.ptr, !llvm.ptr)
+!CHECK-SAME: {{.*}}, omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}
diff --git a/flang/test/Lower/OpenMP/declare-target-deferred-marking.f90 b/flang/test/Lower/OpenMP/declare-target-deferred-marking.f90
index ee2a6c70c0057..a82e2f31afa93 100644
--- a/flang/test/Lower/OpenMP/declare-target-deferred-marking.f90
+++ b/flang/test/Lower/OpenMP/declare-target-deferred-marking.f90
@@ -53,7 +53,7 @@ end program main
!HOST-LABEL: llvm.func @host_interface()
!HOST-SAME: {{.*}}, omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (enter), automap = false>{{.*}}
!ALL-LABEL: llvm.func @called_from_target_interface(!llvm.ptr, !llvm.ptr)
-!ALL-SAME: {{.*}}, omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}
+!ALL-SAME: {{.*}}, omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}
!ALL-LABEL: llvm.func @any_interface()
!ALL-SAME: {{.*}}, omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (enter), automap = false>{{.*}}
!ALL-LABEL: llvm.func @device_interface()
diff --git a/flang/test/Lower/OpenMP/declare-target-implicit-func-and-subr-cap-enter.f90 b/flang/test/Lower/OpenMP/declare-target-implicit-func-and-subr-cap-enter.f90
index 3e20e7a8b0170..26a458e34c950 100644
--- a/flang/test/Lower/OpenMP/declare-target-implicit-func-and-subr-cap-enter.f90
+++ b/flang/test/Lower/OpenMP/declare-target-implicit-func-and-subr-cap-enter.f90
@@ -3,109 +3,134 @@
!RUN: bbc -emit-hlfir -fopenmp -fopenmp-version=52 %s -o - | tco -test-gen | FileCheck %s
!RUN: bbc -emit-hlfir -fopenmp -fopenmp-version=52 -fopenmp-is-target-device %s -o - | tco -test-gen | FileCheck %s --check-prefix=DEVICE
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (enter), automap = false>{{.*}}}
-function implicitly_captured_twice() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_twice
-
-! CHECK-LABEL: llvm.func @_QPtarget_function_twice_host
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (enter), automap = false>{{.*}}}
-function target_function_twice_host() result(i)
-!$omp declare target enter(target_function_twice_host) device_type(host)
- integer :: i
- i = implicitly_captured_twice()
-end function target_function_twice_host
+program mb
+ interface
+ subroutine caller_recursive
+ !$omp declare target enter(caller_recursive) device_type(nohost)
+ end subroutine
-! DEVICE-LABEL: llvm.func @_QPtarget_function_twice_device
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
-function target_function_twice_device() result(i)
-!$omp declare target enter(target_function_twice_device) device_type(nohost)
- integer :: i
- i = implicitly_captured_twice()
-end function target_function_twice_device
+ recursive subroutine implicitly_captured_recursive(increment)
+ integer :: increment
+ end subroutine
+ end interface
-!! -----
+ integer :: tmp
+
+ ! Make sure to make all internal functions reachable. Otherwise, they could
+ ! be optimized out.
+ tmp = target_function_twice_host()
+ tmp = target_function_test_host()
+ !$omp target
+ tmp = target_function_twice_device()
+ tmp = target_function_test()
+ tmp = target_function_test_device()
+ !$omp end target
+
+ contains
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_twice
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_twice() result(k)
+ integer :: i
+ i = 10
+ k = i
+ end function implicitly_captured_twice
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_nest
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
-function implicitly_captured_nest() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_nest
-
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_one
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter){{.*}}}
-function implicitly_captured_one() result(k)
- k = implicitly_captured_nest()
-end function implicitly_captured_one
-
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_two
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
-function implicitly_captured_two() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_two
-
-! DEVICE-LABEL: llvm.func @_QPtarget_function_test
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
-function target_function_test() result(j)
-!$omp declare target enter(target_function_test) device_type(nohost)
- integer :: i, j
- i = implicitly_captured_one()
- j = implicitly_captured_two() + i
-end function target_function_test
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPtarget_function_twice_host()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (enter), automap = false>{{.*}}}
+ function target_function_twice_host() result(i)
+ !$omp declare target enter(target_function_twice_host) device_type(host)
+ integer :: i
+ i = implicitly_captured_twice()
+ end function target_function_twice_host
-!! -----
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPtarget_function_twice_device()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
+ function target_function_twice_device() result(i)
+ !$omp declare target enter(target_function_twice_device) device_type(nohost)
+ integer :: i
+ i = implicitly_captured_twice()
+ end function target_function_twice_device
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_nest_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (enter), automap = false>{{.*}}}
-function implicitly_captured_nest_twice() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_nest_twice
-
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_one_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (enter), automap = false>{{.*}}}
-function implicitly_captured_one_twice() result(k)
- k = implicitly_captured_nest_twice()
-end function implicitly_captured_one_twice
-
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_two_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (enter), automap = false>{{.*}}}
-function implicitly_captured_two_twice() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_two_twice
-
-! DEVICE-LABEL: llvm.func @_QPtarget_function_test_device
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
-function target_function_test_device() result(j)
- !$omp declare target enter(target_function_test_device) device_type(nohost)
- integer :: i, j
- i = implicitly_captured_one_twice()
- j = implicitly_captured_two_twice() + i
-end function target_function_test_device
-
-! CHECK-LABEL: llvm.func @_QPtarget_function_test_host
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (enter), automap = false>{{.*}}}
-function target_function_test_host() result(j)
- !$omp declare target enter(target_function_test_host) device_type(host)
- integer :: i, j
- i = implicitly_captured_one_twice()
- j = implicitly_captured_two_twice() + i
-end function target_function_test_host
+ !! -----
+
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_nest()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_nest() result(k)
+ integer :: i
+ i = 10
+ k = i
+ end function implicitly_captured_nest
+
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_one()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to){{.*}}}
+ function implicitly_captured_one() result(k)
+ k = implicitly_captured_nest()
+ end function implicitly_captured_one
+
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_two()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_two() result(k)
+ integer :: i
+ i = 10
+ k = i
+ end function implicitly_captured_two
+
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPtarget_function_test()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
+ function target_function_test() result(j)
+ !$omp declare target enter(target_function_test) device_type(nohost)
+ integer :: i, j
+ i = implicitly_captured_one()
+ j = implicitly_captured_two() + i
+ end function target_function_test
+
+ !! -----
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_nest_twice()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_nest_twice() result(k)
+ integer :: i
+ i = 10
+ k = i
+ end function implicitly_captured_nest_twice
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_one_twice()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_one_twice() result(k)
+ k = implicitly_captured_nest_twice()
+ end function implicitly_captured_one_twice
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_two_twice()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_two_twice() result(k)
+ integer :: i
+ i = 10
+ k = i
+ end function implicitly_captured_two_twice
+
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPtarget_function_test_device()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
+ function target_function_test_device() result(j)
+ !$omp declare target enter(target_function_test_device) device_type(nohost)
+ integer :: i, j
+ i = implicitly_captured_one_twice()
+ j = implicitly_captured_two_twice() + i
+ end function target_function_test_device
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPtarget_function_test_host()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (enter), automap = false>{{.*}}}
+ function target_function_test_host() result(j)
+ !$omp declare target enter(target_function_test_host) device_type(host)
+ integer :: i, j
+ i = implicitly_captured_one_twice()
+ j = implicitly_captured_two_twice() + i
+ end function target_function_test_host
+
+end program mb
!! -----
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_with_dev_type_recursive
-! DEVICE-SAME: {{.*}}attributes {{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (enter), automap = false>{{.*}}}
+! DEVICE-NOT: llvm.func @_QPimplicitly_captured_with_dev_type_recursive()
recursive function implicitly_captured_with_dev_type_recursive(increment) result(k)
!$omp declare target enter(implicitly_captured_with_dev_type_recursive) device_type(host)
integer :: increment, k
@@ -116,8 +141,8 @@ recursive function implicitly_captured_with_dev_type_recursive(increment) result
end if
end function implicitly_captured_with_dev_type_recursive
-! DEVICE-LABEL: llvm.func @_QPtarget_function_with_dev_type_recurse
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
+! DEVICE-LABEL: llvm.func @_QPtarget_function_with_dev_type_recurse()
+! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
function target_function_with_dev_type_recurse() result(i)
!$omp declare target enter(target_function_with_dev_type_recurse) device_type(nohost)
integer :: i
@@ -128,29 +153,29 @@ end function target_function_with_dev_type_recurse
module test_module
contains
-! CHECK-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_nest_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (enter), automap = false>{{.*}}}
+! CHECK-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_nest_twice()
+! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
function implicitly_captured_nest_twice() result(i)
integer :: i
i = 10
end function implicitly_captured_nest_twice
-! CHECK-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_one_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (enter), automap = false>{{.*}}}
+! CHECK-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_one_twice()
+! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (enter), automap = false>{{.*}}}
function implicitly_captured_one_twice() result(k)
!$omp declare target enter(implicitly_captured_one_twice) device_type(host)
k = implicitly_captured_nest_twice()
end function implicitly_captured_one_twice
-! DEVICE-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_two_twice
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
+! DEVICE-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_two_twice()
+! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
function implicitly_captured_two_twice() result(y)
integer :: y
y = 5
end function implicitly_captured_two_twice
-! DEVICE-LABEL: llvm.func @_QMtest_modulePtarget_function_test_device
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
+! DEVICE-LABEL: llvm.func @_QMtest_modulePtarget_function_test_device()
+! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
function target_function_test_device() result(j)
!$omp declare target enter(target_function_test_device) device_type(nohost)
integer :: i, j
@@ -161,20 +186,8 @@ end module test_module
!! -----
-program mb
- interface
- subroutine caller_recursive
- !$omp declare target enter(caller_recursive) device_type(nohost)
- end subroutine
-
- recursive subroutine implicitly_captured_recursive(increment)
- integer :: increment
- end subroutine
- end interface
-end program
-
! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_recursive
-! DEVICE-SAME: {{.*}}attributes {{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
+! DEVICE-SAME: {{.*}}attributes {{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
recursive subroutine implicitly_captured_recursive(increment)
integer :: increment
if (increment == 10) then
@@ -185,7 +198,7 @@ recursive subroutine implicitly_captured_recursive(increment)
end subroutine
! DEVICE-LABEL: llvm.func @_QPcaller_recursive
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
+! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (enter), automap = false>{{.*}}}
subroutine caller_recursive
!$omp declare target enter(caller_recursive) device_type(nohost)
call implicitly_captured_recursive(0)
diff --git a/flang/test/Lower/OpenMP/declare-target-implicit-func-and-subr-cap.f90 b/flang/test/Lower/OpenMP/declare-target-implicit-func-and-subr-cap.f90
index 01599a045dc6b..7fd3ca9b4e32e 100644
--- a/flang/test/Lower/OpenMP/declare-target-implicit-func-and-subr-cap.f90
+++ b/flang/test/Lower/OpenMP/declare-target-implicit-func-and-subr-cap.f90
@@ -3,135 +3,160 @@
!RUN: bbc -emit-hlfir -fopenmp -fopenmp-version=50 %s -o - | tco -test-gen | FileCheck %s
!RUN: bbc -emit-hlfir -fopenmp -fopenmp-version=50 -fopenmp-is-target-device %s -o - | tco -test-gen | FileCheck %s --check-prefix=DEVICE
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured(toggle) result(k)
- integer :: i, j, k
- logical :: toggle
- i = 10
- j = 5
- if (toggle) then
+program mb
+ interface
+ subroutine caller_recursive
+ !$omp declare target to(caller_recursive) device_type(nohost)
+ end subroutine
+
+ recursive subroutine implicitly_captured_recursive(increment)
+ integer :: increment
+ end subroutine
+ end interface
+
+ integer :: tmp
+
+ ! Make sure to make all internal functions reachable. Otherwise, they could
+ ! be optimized out.
+ tmp = target_function(.true.)
+ tmp = target_function_twice_host()
+ tmp = target_function_test_host()
+ !$omp target
+ tmp = target_function_twice_device()
+ tmp = target_function_test()
+ tmp = target_function_test_device()
+ !$omp end target
+
+ contains
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured(
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured(toggle) result(k)
+ integer :: i, j, k
+ logical :: toggle
+ i = 10
+ j = 5
+ if (toggle) then
+ k = i
+ else
+ k = j
+ end if
+ end function implicitly_captured
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPtarget_function(
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function target_function(toggle) result(i)
+ !$omp declare target
+ integer :: i
+ logical :: toggle
+ i = implicitly_captured(toggle)
+ end function target_function
+
+ !! -----
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_twice()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_twice() result(k)
+ integer :: i
+ i = 10
k = i
- else
- k = j
- end if
-end function implicitly_captured
+ end function implicitly_captured_twice
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPtarget_function_twice_host()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>{{.*}}}
+ function target_function_twice_host() result(i)
+ !$omp declare target to(target_function_twice_host) device_type(host)
+ integer :: i
+ i = implicitly_captured_twice()
+ end function target_function_twice_host
-! CHECK-LABEL: llvm.func @_QPtarget_function
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
-function target_function(toggle) result(i)
-!$omp declare target
- integer :: i
- logical :: toggle
- i = implicitly_captured(toggle)
-end function target_function
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPtarget_function_twice_device()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ function target_function_twice_device() result(i)
+ !$omp declare target to(target_function_twice_device) device_type(nohost)
+ integer :: i
+ i = implicitly_captured_twice()
+ end function target_function_twice_device
-!! -----
+ !! -----
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured_twice() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_twice
-
-! CHECK-LABEL: llvm.func @_QPtarget_function_twice_host
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>{{.*}}}
-function target_function_twice_host() result(i)
-!$omp declare target to(target_function_twice_host) device_type(host)
- integer :: i
- i = implicitly_captured_twice()
-end function target_function_twice_host
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_nest()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_nest() result(k)
+ integer :: i
+ i = 10
+ k = i
+ end function implicitly_captured_nest
-! DEVICE-LABEL: llvm.func @_QPtarget_function_twice_device
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-function target_function_twice_device() result(i)
-!$omp declare target to(target_function_twice_device) device_type(nohost)
- integer :: i
- i = implicitly_captured_twice()
-end function target_function_twice_device
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_one()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to){{.*}}}
+ function implicitly_captured_one() result(k)
+ k = implicitly_captured_nest()
+ end function implicitly_captured_one
-!! -----
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_two()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_two() result(k)
+ integer :: i
+ i = 10
+ k = i
+ end function implicitly_captured_two
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_nest
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured_nest() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_nest
-
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_one
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to){{.*}}}
-function implicitly_captured_one() result(k)
- k = implicitly_captured_nest()
-end function implicitly_captured_one
-
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_two
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured_two() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_two
-
-! DEVICE-LABEL: llvm.func @_QPtarget_function_test
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-function target_function_test() result(j)
-!$omp declare target to(target_function_test) device_type(nohost)
- integer :: i, j
- i = implicitly_captured_one()
- j = implicitly_captured_two() + i
-end function target_function_test
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPtarget_function_test()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ function target_function_test() result(j)
+ !$omp declare target to(target_function_test) device_type(nohost)
+ integer :: i, j
+ i = implicitly_captured_one()
+ j = implicitly_captured_two() + i
+ end function target_function_test
-!! -----
+ !! -----
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_nest_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured_nest_twice() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_nest_twice
-
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_one_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured_one_twice() result(k)
- k = implicitly_captured_nest_twice()
-end function implicitly_captured_one_twice
-
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_two_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured_two_twice() result(k)
- integer :: i
- i = 10
- k = i
-end function implicitly_captured_two_twice
-
-! DEVICE-LABEL: llvm.func @_QPtarget_function_test_device
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-function target_function_test_device() result(j)
- !$omp declare target to(target_function_test_device) device_type(nohost)
- integer :: i, j
- i = implicitly_captured_one_twice()
- j = implicitly_captured_two_twice() + i
-end function target_function_test_device
-
-! CHECK-LABEL: llvm.func @_QPtarget_function_test_host
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>{{.*}}}
-function target_function_test_host() result(j)
- !$omp declare target to(target_function_test_host) device_type(host)
- integer :: i, j
- i = implicitly_captured_one_twice()
- j = implicitly_captured_two_twice() + i
-end function target_function_test_host
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_nest_twice()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_nest_twice() result(k)
+ integer :: i
+ i = 10
+ k = i
+ end function implicitly_captured_nest_twice
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_one_twice()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_one_twice() result(k)
+ k = implicitly_captured_nest_twice()
+ end function implicitly_captured_one_twice
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_two_twice()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_two_twice() result(k)
+ integer :: i
+ i = 10
+ k = i
+ end function implicitly_captured_two_twice
+
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPtarget_function_test_device()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ function target_function_test_device() result(j)
+ !$omp declare target to(target_function_test_device) device_type(nohost)
+ integer :: i, j
+ i = implicitly_captured_one_twice()
+ j = implicitly_captured_two_twice() + i
+ end function target_function_test_device
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPtarget_function_test_host()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>{{.*}}}
+ function target_function_test_host() result(j)
+ !$omp declare target to(target_function_test_host) device_type(host)
+ integer :: i, j
+ i = implicitly_captured_one_twice()
+ j = implicitly_captured_two_twice() + i
+ end function target_function_test_host
+
+end program mb
!! -----
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_with_dev_type_recursive
-! DEVICE-SAME: {{.*}}attributes {{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+! DEVICE-NOT: llvm.func @_QPimplicitly_captured_with_dev_type_recursive()
recursive function implicitly_captured_with_dev_type_recursive(increment) result(k)
!$omp declare target to(implicitly_captured_with_dev_type_recursive) device_type(host)
integer :: increment, k
@@ -142,8 +167,8 @@ recursive function implicitly_captured_with_dev_type_recursive(increment) result
end if
end function implicitly_captured_with_dev_type_recursive
-! DEVICE-LABEL: llvm.func @_QPtarget_function_with_dev_type_recurse
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+! DEVICE-LABEL: llvm.func @_QPtarget_function_with_dev_type_recurse()
+! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
function target_function_with_dev_type_recurse() result(i)
!$omp declare target to(target_function_with_dev_type_recurse) device_type(nohost)
integer :: i
@@ -154,29 +179,29 @@ end function target_function_with_dev_type_recurse
module test_module
contains
-! CHECK-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_nest_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+! CHECK-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_nest_twice()
+! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
function implicitly_captured_nest_twice() result(i)
integer :: i
i = 10
end function implicitly_captured_nest_twice
-! CHECK-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_one_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
+! CHECK-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_one_twice()
+! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>{{.*}}}
function implicitly_captured_one_twice() result(k)
!$omp declare target to(implicitly_captured_one_twice) device_type(host)
k = implicitly_captured_nest_twice()
end function implicitly_captured_one_twice
-! DEVICE-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_two_twice
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+! DEVICE-LABEL: llvm.func @_QMtest_modulePimplicitly_captured_two_twice()
+! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
function implicitly_captured_two_twice() result(y)
integer :: y
y = 5
end function implicitly_captured_two_twice
-! DEVICE-LABEL: llvm.func @_QMtest_modulePtarget_function_test_device
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+! DEVICE-LABEL: llvm.func @_QMtest_modulePtarget_function_test_device()
+! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
function target_function_test_device() result(j)
!$omp declare target to(target_function_test_device) device_type(nohost)
integer :: i, j
@@ -187,20 +212,8 @@ end module test_module
!! -----
-program mb
- interface
- subroutine caller_recursive
- !$omp declare target to(caller_recursive) device_type(nohost)
- end subroutine
-
- recursive subroutine implicitly_captured_recursive(increment)
- integer :: increment
- end subroutine
- end interface
-end program
-
! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_recursive
-! DEVICE-SAME: {{.*}}attributes {{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+! DEVICE-SAME: {{.*}}attributes {{.*}}omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
recursive subroutine implicitly_captured_recursive(increment)
integer :: increment
if (increment == 10) then
@@ -211,7 +224,7 @@ recursive subroutine implicitly_captured_recursive(increment)
end subroutine
! DEVICE-LABEL: llvm.func @_QPcaller_recursive
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
subroutine caller_recursive
!$omp declare target to(caller_recursive) device_type(nohost)
call implicitly_captured_recursive(0)
diff --git a/flang/test/Lower/OpenMP/declare-target-implicit-tarop-cap.f90 b/flang/test/Lower/OpenMP/declare-target-implicit-tarop-cap.f90
index 7a04cedf22474..eca6e5ba39267 100644
--- a/flang/test/Lower/OpenMP/declare-target-implicit-tarop-cap.f90
+++ b/flang/test/Lower/OpenMP/declare-target-implicit-tarop-cap.f90
@@ -3,83 +3,96 @@
!RUN: bbc -emit-hlfir -fopenmp -fopenmp-version=52 %s -o - | tco -test-gen | FileCheck %s
!RUN: bbc -emit-hlfir -fopenmp -fopenmp-version=52 -fopenmp-is-target-device %s -o - | tco -test-gen | FileCheck %s --check-prefix=DEVICE
-! DEVICE-LABEL: llvm.func @_QPimplicit_capture
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-function implicit_capture() result(i)
- implicit none
- integer :: i
- i = 1
-end function implicit_capture
-
-subroutine subr_target()
- integer :: n
-!$omp target map(tofrom:n)
- n = implicit_capture()
-!$omp end target
-end subroutine
-
-!! -----
-
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_nest_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured_nest_twice() result(i)
- integer :: i
- i = 10
-end function implicitly_captured_nest_twice
-
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_one_twice
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured_one_twice() result(k)
-!$omp declare target to(implicitly_captured_one_twice) device_type(host)
- k = implicitly_captured_nest_twice()
-end function implicitly_captured_one_twice
-
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_nest_twice_enter
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (enter), automap = false>{{.*}}}
-function implicitly_captured_nest_twice_enter() result(i)
- integer :: i
- i = 10
-end function implicitly_captured_nest_twice_enter
-
-! CHECK-LABEL: llvm.func @_QPimplicitly_captured_one_twice_enter
-! CHECK-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (enter), automap = false>{{.*}}}
-function implicitly_captured_one_twice_enter() result(k)
-!$omp declare target enter(implicitly_captured_one_twice_enter) device_type(host)
- k = implicitly_captured_nest_twice_enter()
-end function implicitly_captured_one_twice_enter
-
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_two_twice
-! DEVICE-SAME: {{.*}}attributes {omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-function implicitly_captured_two_twice() result(y)
- integer :: y
- y = 5
-end function implicitly_captured_two_twice
-
-
-function target_function_test_device() result(j)
- integer :: i, j
- !$omp target map(tofrom: i, j)
- i = implicitly_captured_one_twice()
- j = implicitly_captured_two_twice() + i
- !$omp end target
-end function target_function_test_device
-
-!! -----
-
-! DEVICE-LABEL: llvm.func @_QPimplicitly_captured_recursive
-! DEVICE-SAME: {{.*}}attributes {{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
-recursive function implicitly_captured_recursive(increment) result(k)
- integer :: increment, k
- if (increment == 10) then
- k = increment
- else
- k = implicitly_captured_recursive(increment + 1)
- end if
-end function implicitly_captured_recursive
-
-function target_function_recurse() result(i)
- integer :: i
- !$omp target map(tofrom: i)
- i = implicitly_captured_recursive(0)
+program main
+ integer :: tmp
+
+ ! Make sure to make all internal functions reachable. Otherwise, they could
+ ! be optimized out.
+ call subr_target()
+ tmp = implicitly_captured_one_twice_enter()
+ tmp = target_function_test_device()
+ tmp = target_function_recurse()
+
+ contains
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPimplicit_capture()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ function implicit_capture() result(i)
+ implicit none
+ integer :: i
+ i = 1
+ end function implicit_capture
+
+ subroutine subr_target()
+ integer :: n
+ !$omp target map(tofrom:n)
+ n = implicit_capture()
!$omp end target
-end function target_function_recurse
+ end subroutine
+
+ !! -----
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_nest_twice()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_nest_twice() result(i)
+ integer :: i
+ i = 10
+ end function implicitly_captured_nest_twice
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_one_twice()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_one_twice() result(k)
+ !$omp declare target to(implicitly_captured_one_twice) device_type(host)
+ k = implicitly_captured_nest_twice()
+ end function implicitly_captured_one_twice
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_nest_twice_enter()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_nest_twice_enter() result(i)
+ integer :: i
+ i = 10
+ end function implicitly_captured_nest_twice_enter
+
+ ! CHECK-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_one_twice_enter()
+ ! CHECK-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (enter), automap = false>{{.*}}}
+ function implicitly_captured_one_twice_enter() result(k)
+ !$omp declare target enter(implicitly_captured_one_twice_enter) device_type(host)
+ k = implicitly_captured_nest_twice_enter()
+ end function implicitly_captured_one_twice_enter
+
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_two_twice()
+ ! DEVICE-SAME: {{.*}}attributes {{{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ function implicitly_captured_two_twice() result(y)
+ integer :: y
+ y = 5
+ end function implicitly_captured_two_twice
+
+
+ function target_function_test_device() result(j)
+ integer :: i, j
+ !$omp target map(tofrom: i, j)
+ i = implicitly_captured_one_twice()
+ j = implicitly_captured_two_twice() + i
+ !$omp end target
+ end function target_function_test_device
+
+ !! -----
+
+ ! DEVICE-LABEL: llvm.func{{.*}} @_QFPimplicitly_captured_recursive(
+ ! DEVICE-SAME: {{.*}}attributes {{.*}}omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>{{.*}}}
+ recursive function implicitly_captured_recursive(increment) result(k)
+ integer :: increment, k
+ if (increment == 10) then
+ k = increment
+ else
+ k = implicitly_captured_recursive(increment + 1)
+ end if
+ end function implicitly_captured_recursive
+
+ function target_function_recurse() result(i)
+ integer :: i
+ !$omp target map(tofrom: i)
+ i = implicitly_captured_recursive(0)
+ !$omp end target
+ end function target_function_recurse
+
+end program main
diff --git a/flang/test/Lower/OpenMP/declare-target-named-main-interface.f90 b/flang/test/Lower/OpenMP/declare-target-named-main-interface.f90
index f002de46d721b..19d245225d53d 100644
--- a/flang/test/Lower/OpenMP/declare-target-named-main-interface.f90
+++ b/flang/test/Lower/OpenMP/declare-target-named-main-interface.f90
@@ -4,11 +4,11 @@
! Test that a bare '!$omp declare target' inside an interface body that
! appears in a *named* main program does not incorrectly mark the main
! program (_QQmain) as a declare-target function while still correctly
-! marking the declared subroutine (sub_a) as device_type(nohost).
+! marking the declared subroutine (sub_a) as device_type(any).
! CHECK-NOT: llvm.func @_QQmain{{.*}}device_type = (any)
! CHECK-NOT: llvm.func @_QQmain{{.*}}device_type = (nohost)
-! CHECK: llvm.func @_QPsub_a{{.*}}device_type = (nohost), {{.*}}sym_visibility = "private"
+! CHECK: llvm.func @_QPsub_a{{.*}}device_type = (any), {{.*}}sym_visibility = "private"
program named_main
interface
diff --git a/mlir/include/mlir/Dialect/OpenMP/Transforms/Passes.td b/mlir/include/mlir/Dialect/OpenMP/Transforms/Passes.td
index 3825eeb286e72..217c299a9a0e1 100644
--- a/mlir/include/mlir/Dialect/OpenMP/Transforms/Passes.td
+++ b/mlir/include/mlir/Dialect/OpenMP/Transforms/Passes.td
@@ -48,6 +48,9 @@ def MarkDeclareTargetPass : Pass<"omp-mark-declare-target", "ModuleOp"> {
Marks functions contained within the module as declare target if they are
called from within an explicitly marked declare target function or a target
region (omp.target).
+
+ Also, mark transitively reached functions through recipe ops (e.g.
+ omp.private) and other function calls.
}];
let dependentDialects = ["mlir::omp::OpenMPDialect"];
}
diff --git a/mlir/lib/Dialect/OpenMP/Transforms/MarkDeclareTarget.cpp b/mlir/lib/Dialect/OpenMP/Transforms/MarkDeclareTarget.cpp
index 84b0d40949dba..138f36f74f644 100644
--- a/mlir/lib/Dialect/OpenMP/Transforms/MarkDeclareTarget.cpp
+++ b/mlir/lib/Dialect/OpenMP/Transforms/MarkDeclareTarget.cpp
@@ -16,7 +16,8 @@
#include "mlir/Interfaces/FunctionInterfaces.h"
#include "mlir/Pass/Pass.h"
#include "mlir/Support/LLVM.h"
-#include "llvm/ADT/SmallPtrSet.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringSet.h"
#include "llvm/ADT/TypeSwitch.h"
namespace mlir {
@@ -29,143 +30,285 @@ namespace omp {
} // namespace mlir
using namespace mlir;
-namespace {
-class MarkDeclareTargetPass
- : public omp::impl::MarkDeclareTargetPassBase<MarkDeclareTargetPass> {
+/// Check whether the given operation is located inside of an \c omp.target.
+static bool isInTargetRegion(Operation &op) {
+ // TODO: Detection of callees inside of a target region might need an update
+ // once reverse offloading is implemented.
+ // Reverse offload target regions would then have to propagate the "host"
+ // device type.
+ return op.getParentOfType<omp::TargetOp>();
+}
- struct ParentInfo {
- omp::DeclareTargetDeviceType devTy;
- omp::DeclareTargetCaptureClause capClause;
- bool automap;
- };
-
- void processSymbolRef(SymbolRefAttr symRef, ParentInfo parentInfo,
- llvm::SmallPtrSet<Operation *, 16> visited) {
- Operation *symOp = getOperation().lookupSymbol(symRef);
- if (!symOp)
- return;
- auto current = llvm::dyn_cast<omp::DeclareTargetInterface>(symOp);
- if (!current)
- return;
-
- if (current.isDeclareTarget()) {
- auto currentDt = current.getDeclareTargetDeviceType();
-
- // Found the same function twice, with different device_types,
- // mark as Any as it belongs to both
- if (currentDt != parentInfo.devTy &&
- currentDt != omp::DeclareTargetDeviceType::any) {
- current.setDeclareTarget(omp::DeclareTargetDeviceType::any,
- current.getDeclareTargetCaptureClause(),
- current.getDeclareTargetAutomap());
- }
- } else {
- current.setDeclareTarget(parentInfo.devTy, parentInfo.capClause,
- parentInfo.automap);
+/// Add to \c callees all names of the functions called from regions owned by
+/// \c op. If \c targetCallees is provided, split non-target and target uses
+/// between these two output sets.
+static void gatherNestedCallees(Operation &op, llvm::StringSet<> &callees,
+ llvm::StringSet<> *targetCallees = nullptr) {
+ op.walk([&](CallOpInterface callOp) {
+ CallInterfaceCallable callable = callOp.getCallableForCallee();
+ if (auto callableSymRef = dyn_cast<SymbolRefAttr>(callable)) {
+ StringRef callee = callableSymRef.getLeafReference();
+ if (targetCallees && isInTargetRegion(*callOp))
+ targetCallees->insert(callee);
+ else
+ callees.insert(callee);
}
+ });
+}
- markNestedFuncs(parentInfo, symOp, visited);
- }
+/// Extract from \c arrayAttr and into \c syms the list of symbol names stored
+/// in the attribute.
+static void gatherSymsFromAttr(ArrayAttr arrayAttr, llvm::StringSet<> &syms) {
+ if (!arrayAttr)
+ return;
- void processReductionRefs(std::optional<mlir::ArrayAttr> symRefs,
- ParentInfo parentInfo,
- llvm::SmallPtrSet<Operation *, 16> visited) {
- if (!symRefs)
- return;
+ for (Attribute attr : arrayAttr)
+ if (auto symbolRefAttr = dyn_cast<SymbolRefAttr>(attr))
+ syms.insert(symbolRefAttr.getLeafReference());
+}
- for (auto symRef : symRefs->getAsRange<mlir::SymbolRefAttr>()) {
- if (auto declareReductionOp =
- getOperation().lookupSymbol<omp::DeclareReductionOp>(symRef)) {
- markNestedFuncs(parentInfo, declareReductionOp, visited);
- }
- }
- }
+/// Go through all OpenMP dialect operations located in regions owned by \c op
+/// looking for symbol references to \c accomp::RecipeInterface or
+/// \c FunctionOpInterface operations and, based on whether they are located
+/// within a nested \c omp.target region, add them to the corresponding output
+/// \c StringSet.
+static void gatherNestedSymbolUses(Operation &op,
+ llvm::StringSet<> &nestedRecipeUses,
+ llvm::StringSet<> &targetRecipeUses,
+ llvm::StringSet<> &nestedFunctionUses,
+ llvm::StringSet<> &targetFunctionUses) {
+ op.walk([&](Operation *op) {
+ bool inTarget = isInTargetRegion(*op);
+ llvm::StringSet<> &recipeUses =
+ inTarget ? targetRecipeUses : nestedRecipeUses;
+ llvm::StringSet<> &functionUses =
+ inTarget ? targetFunctionUses : nestedFunctionUses;
- void processReductionClauses(Operation *op, ParentInfo parentInfo,
- llvm::SmallPtrSet<Operation *, 16> visited) {
+ // Handle each op holding clauses linked to a recipe op separately. This
+ // must be kept in sync with dialect changes.
llvm::TypeSwitch<Operation &>(*op)
+ .Case([&](omp::DistributeOp op) {
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ })
.Case([&](omp::LoopOp op) {
- processReductionRefs(op.getReductionSyms(), parentInfo, visited);
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getReductionSymsAttr(), recipeUses);
+ })
+ .Case([&](omp::MapInfoOp op) {
+ if (FlatSymbolRefAttr mapperAttr = op.getMapperIdAttr())
+ recipeUses.insert(mapperAttr.getValue());
})
.Case([&](omp::ParallelOp op) {
- processReductionRefs(op.getReductionSyms(), parentInfo, visited);
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getReductionSymsAttr(), recipeUses);
+ })
+ .Case([&](omp::ScopeOp op) {
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getReductionSymsAttr(), recipeUses);
})
.Case([&](omp::SectionsOp op) {
- processReductionRefs(op.getReductionSyms(), parentInfo, visited);
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getReductionSymsAttr(), recipeUses);
})
.Case([&](omp::SimdOp op) {
- processReductionRefs(op.getReductionSyms(), parentInfo, visited);
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getReductionSymsAttr(), recipeUses);
+ })
+ .Case([&](omp::SingleOp op) {
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ // This goes directly to the called functions, as it's pointing to a
+ // function, not a recipe op.
+ gatherSymsFromAttr(op.getCopyprivateSymsAttr(), functionUses);
})
.Case([&](omp::TargetOp op) {
- processReductionRefs(op.getInReductionSyms(), parentInfo, visited);
+ // omp.private is inlined inside of the target region, hence we need
+ // to add it with the target uses rather than base it on context.
+ // TODO: The reverse-offload case would require adding it to
+ // nestedRecipeUses.
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), targetRecipeUses);
+ gatherSymsFromAttr(op.getInReductionSymsAttr(), recipeUses);
})
.Case([&](omp::TaskgroupOp op) {
- processReductionRefs(op.getTaskReductionSyms(), parentInfo, visited);
+ gatherSymsFromAttr(op.getTaskReductionSymsAttr(), recipeUses);
})
.Case([&](omp::TaskloopContextOp op) {
- processReductionRefs(op.getReductionSyms(), parentInfo, visited);
- processReductionRefs(op.getInReductionSyms(), parentInfo, visited);
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getReductionSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getInReductionSymsAttr(), recipeUses);
})
.Case([&](omp::TaskOp op) {
- processReductionRefs(op.getInReductionSyms(), parentInfo, visited);
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getInReductionSymsAttr(), recipeUses);
})
.Case([&](omp::TeamsOp op) {
- processReductionRefs(op.getReductionSyms(), parentInfo, visited);
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getReductionSymsAttr(), recipeUses);
})
.Case([&](omp::WsloopOp op) {
- processReductionRefs(op.getReductionSyms(), parentInfo, visited);
- })
- .Default([](Operation &) {});
- }
+ gatherSymsFromAttr(op.getPrivateSymsAttr(), recipeUses);
+ gatherSymsFromAttr(op.getReductionSymsAttr(), recipeUses);
+ });
+ });
+}
- void markNestedFuncs(ParentInfo parentInfo, Operation *currOp,
- llvm::SmallPtrSet<Operation *, 16> visited) {
- if (visited.contains(currOp))
- return;
- visited.insert(currOp);
-
- currOp->walk([&, this](Operation *op) {
- if (auto callOp = llvm::dyn_cast<CallOpInterface>(op)) {
- if (auto symRef = llvm::dyn_cast_if_present<mlir::SymbolRefAttr>(
- callOp.getCallableForCallee())) {
- processSymbolRef(symRef, parentInfo, visited);
- }
- }
- processReductionClauses(op, parentInfo, visited);
- });
- }
+namespace {
+
+// If this pass runs more than once, something like this can happen:
+// - 1st run: The pass marks an external function as declare_target with
+// device_type(nohost) based on there being a single call from an omp.target.
+// - Somewhere in between: New calls to that external function are added to the
+// host part of the application (e.g. it is part of a standard library).
+// - 2nd run: The pass doesn't update the function after seeing it's reachable
+// by the host because it's unable to tell that the declare_target information
+// wasn't explicitly added by the user.
+// TODO: This can be fixed by adding a discardable attribute only used by this
+// pass or by extending the DeclareTargetInterface to also store whether it is
+// implicit or explicit.
+class MarkDeclareTargetPass
+ : public omp::impl::MarkDeclareTargetPassBase<MarkDeclareTargetPass> {
- // This pass executes on mlir::ModuleOp's marking functions contained within
+ // This pass executes on mlir::ModuleOp, marking functions contained within
// as implicitly declare target if they are called from within an explicitly
- // marked declare target function or a target region (TargetOp)
+ // marked declare target function or a target region (TargetOp), or
+ // transitively through recipe ops (e.g. omp.declare_reduction, omp.private)
+ // or other function calls.
void runOnOperation() override {
+ // Illegal as an MLIR symbol name to avoid collisions. Used to gather all
+ // calls from within omp.target regions as a single "function".
+ constexpr const static ::llvm::StringLiteral kTargetRegionsSymName =
+ "omp targets";
+
+ ModuleOp modOp = getOperation();
+
+ // Gather and store the set of called functions by each recipe.
+ // TODO: This doesn't currently support recipe ops holding references to
+ // other recipe ops.
+ llvm::StringMap<llvm::StringSet<>> calls;
+ for (auto recipeOp : modOp.getOps<accomp::RecipeInterface>()) {
+ StringAttr recipeSymName;
+ if (auto symOp = dyn_cast<SymbolOpInterface>(*recipeOp))
+ recipeSymName = symOp.getNameAttr();
+ else if (auto privateOp = dyn_cast<omp::PrivateClauseOp>(*recipeOp))
+ recipeSymName = privateOp.getSymNameAttr();
+
+ if (recipeSymName) {
+ llvm::StringSet<> recipeCalls;
+ gatherNestedCallees(*recipeOp, recipeCalls);
+ calls[recipeSymName] = recipeCalls;
+ }
+ }
+
+ // Gather and store the set of called functions by each function.
+ for (auto funcOp : modOp.getOps<FunctionOpInterface>()) {
+ llvm::StringSet<> functionCalls, targetCalls;
+ gatherNestedCallees(*funcOp, functionCalls, &targetCalls);
+
+ // Transitively include functions called from recipe op users, as if
+ // inlined.
+ llvm::StringSet<> recipeUses, targetRecipeUses;
+ gatherNestedSymbolUses(*funcOp, recipeUses, targetRecipeUses,
+ functionCalls, targetCalls);
+ for (auto &recipe : recipeUses) {
+ const llvm::StringSet<> &recipeCalls = calls.at(recipe.getKey());
+ functionCalls.insert_range(recipeCalls);
+ }
+ for (auto &recipe : targetRecipeUses) {
+ const llvm::StringSet<> &recipeCalls = calls.at(recipe.getKey());
+ targetCalls.insert_range(recipeCalls);
+ }
+
+ calls[funcOp.getName()] = functionCalls;
+ calls[kTargetRegionsSymName].insert_range(targetCalls);
+ }
+
+ // Create worklist with all functions that are directly reachable from
+ // declare_target functions or target regions.
+ llvm::SmallVector<std::pair<StringRef, omp::DeclareTargetDeviceType>>
+ worklist;
for (auto funcOp : getOperation().getOps<FunctionOpInterface>()) {
auto declareTargetOp =
llvm::dyn_cast<omp::DeclareTargetInterface>(funcOp.getOperation());
+
if (!declareTargetOp || !declareTargetOp.isDeclareTarget())
continue;
- llvm::SmallPtrSet<Operation *, 16> visited;
- ParentInfo parentInfo{declareTargetOp.getDeclareTargetDeviceType(),
- declareTargetOp.getDeclareTargetCaptureClause(),
- declareTargetOp.getDeclareTargetAutomap()};
- markNestedFuncs(parentInfo, funcOp, visited);
+
+ // Add to the worklist all called functions with the declare_target
+ // information of this one, so it gets propagated.
+ for (auto &callee : calls[funcOp.getName()])
+ worklist.push_back(
+ {callee.getKey(), declareTargetOp.getDeclareTargetDeviceType()});
}
- // TODO: Extend to work with reverse-offloading, this shouldn't
- // require too much effort, just need to check the device clause
- // when it's lowering has been implemented and change the
- // DeclareTargetDeviceType argument from nohost to host depending on
- // the contents of the device clause
- getOperation()->walk([&](omp::TargetOp tarOp) {
- llvm::SmallPtrSet<Operation *, 16> visited;
- ParentInfo parentInfo = {
- /*devTy=*/omp::DeclareTargetDeviceType::nohost,
- /*capClause=*/omp::DeclareTargetCaptureClause::to,
- /*automap=*/false,
- };
- markNestedFuncs(parentInfo, tarOp, visited);
- });
+ // Add to the worklist all functions reached from target regions.
+ for (auto &callee : calls[kTargetRegionsSymName])
+ worklist.push_back(
+ {callee.getKey(), omp::DeclareTargetDeviceType::nohost});
+
+ // Process the work list storing intermediate declare target information
+ // separately to avoid mixing up explicit declare_target functions with
+ // implicitly propagated information.
+ llvm::StringMap<omp::DeclareTargetDeviceType> intermediateInfos;
+ while (!worklist.empty()) {
+ std::pair<StringRef, omp::DeclareTargetDeviceType> workItem =
+ worklist.pop_back_val();
+ auto funcOp = modOp.lookupSymbol<FunctionOpInterface>(workItem.first);
+ assert(funcOp && "a work item must point to an existing function");
+
+ // Skip if the function is explicitly marked as declare_target or if it
+ // doesn't support the interface. We only want to propagate implicit
+ // declare_target information to functions for which the user hasn't
+ // specified an explicit behavior.
+ if (auto declareTargetOp =
+ dyn_cast<omp::DeclareTargetInterface>(*funcOp)) {
+ if (declareTargetOp.isDeclareTarget())
+ continue;
+ } else {
+ continue;
+ }
+
+ omp::DeclareTargetDeviceType changedDeviceType;
+ if (!intermediateInfos.contains(workItem.first)) {
+ // Prevent public and external functions from being restricted to a
+ // device. We don't have visibility over all their uses.
+ if (funcOp.isPublic() || funcOp.isExternal())
+ changedDeviceType = omp::DeclareTargetDeviceType::any;
+ else
+ changedDeviceType = workItem.second;
+
+ intermediateInfos.try_emplace(workItem.first, changedDeviceType);
+ } else {
+ omp::DeclareTargetDeviceType ¤tDeviceType =
+ intermediateInfos[workItem.first];
+
+ // Skip the update (and adding callees to the worklist) if the added
+ // info doesn't change anything.
+ if (currentDeviceType == omp::DeclareTargetDeviceType::any ||
+ currentDeviceType == workItem.second) {
+ continue;
+ }
+
+ // Update intermediate information about this function. By the previous
+ // check, we know it's host + nohost = any.
+ changedDeviceType = currentDeviceType =
+ omp::DeclareTargetDeviceType::any;
+ }
+
+ // Add callees to the worklist to propagate the update.
+ for (auto &callee : calls[workItem.first])
+ worklist.push_back({callee.getKey(), changedDeviceType});
+ }
+
+ // Apply the final intermediate results to the corresponding operations.
+ for (auto &[funcName, deviceType] : intermediateInfos) {
+ auto declareTargetOp =
+ modOp.lookupSymbol<omp::DeclareTargetInterface>(funcName);
+ assert(declareTargetOp &&
+ "declare_target info attached to incompatible operation");
+ declareTargetOp.setDeclareTarget(deviceType,
+ omp::DeclareTargetCaptureClause::to,
+ /*automap=*/false);
+ }
}
};
} // namespace
diff --git a/mlir/test/Dialect/OpenMP/mark-declare-target.mlir b/mlir/test/Dialect/OpenMP/mark-declare-target.mlir
new file mode 100644
index 0000000000000..389510c4eedb4
--- /dev/null
+++ b/mlir/test/Dialect/OpenMP/mark-declare-target.mlir
@@ -0,0 +1,363 @@
+// RUN: mlir-opt --split-input-file --omp-mark-declare-target %s | FileCheck %s
+
+// declare_target information gets propagated across omp.private.
+
+omp.private {type = firstprivate} @priv : !llvm.struct<(ptr)> init {
+^bb0(%arg0: !llvm.ptr, %arg1: !llvm.ptr):
+ llvm.call @priv_callee_init() : () -> ()
+ omp.yield(%arg1 : !llvm.ptr)
+} copy {
+^bb0(%arg0: !llvm.ptr, %arg1: !llvm.ptr):
+ llvm.call @priv_callee_copy() : () -> ()
+ omp.yield(%arg1 : !llvm.ptr)
+} dealloc {
+^bb0(%arg0: !llvm.ptr):
+ llvm.call @priv_callee_dealloc() : () -> ()
+ omp.yield
+}
+
+// CHECK: llvm.func {{.*}}@priv_callee_nested()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @priv_callee_nested() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@priv_callee_init()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @priv_callee_init() attributes {sym_visibility = "private"} {
+ llvm.call @priv_callee_nested() : () -> ()
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@priv_callee_copy()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @priv_callee_copy() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@priv_callee_dealloc()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @priv_callee_dealloc() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@main()
+// CHECK-NOT: omp.declare_target
+// CHECK: llvm.return
+llvm.func @main() {
+ omp.target kernel_type(generic) {
+ %0 = llvm.mlir.constant(1 : i64) : i64
+ %1 = llvm.alloca %0 x i32 : (i64) -> !llvm.ptr
+ omp.parallel private(@priv %1 -> %arg0 : !llvm.ptr) {
+ omp.terminator
+ }
+ omp.terminator
+ }
+ llvm.return
+}
+
+// -----
+
+// declare_target information gets propagated across omp.declare_reduction.
+
+omp.declare_reduction @red : i32
+init {
+^bb0(%arg0: i32):
+ llvm.call @red_callee_init() : () -> ()
+ omp.yield (%arg0 : i32)
+}
+combiner {
+^bb1(%arg0: i32, %arg1: i32):
+ llvm.call @red_callee_combiner() : () -> ()
+ omp.yield (%arg0 : i32)
+}
+cleanup {
+^bb0(%arg0: i32):
+ llvm.call @red_callee_cleanup() : () -> ()
+ omp.yield
+}
+
+// CHECK: llvm.func {{.*}}@red_callee_nested()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>
+llvm.func @red_callee_nested() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@red_callee_init()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>
+llvm.func @red_callee_init() attributes {sym_visibility = "private"} {
+ llvm.call @red_callee_nested() : () -> ()
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@red_callee_combiner()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>
+llvm.func @red_callee_combiner() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@red_callee_cleanup()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>
+llvm.func @red_callee_cleanup() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@main(
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>
+llvm.func @main(%arg0 : !llvm.ptr) attributes {
+ omp.declare_target = #omp.declaretarget<
+ device_type = (host), capture_clause = (to), automap = false>} {
+ omp.parallel reduction(@red %arg0 -> %arg1 : !llvm.ptr) {
+ omp.terminator
+ }
+ llvm.return
+}
+
+// -----
+
+// declare_target information gets propagated across the sequence:
+// declare_target fn -> fn -> private -> fn -> reduction -> fn -> fn.
+
+omp.private {type = firstprivate} @priv : !llvm.struct<(ptr)> init {
+^bb0(%arg0: !llvm.ptr, %arg1: !llvm.ptr):
+ llvm.call @priv_callee() : () -> ()
+ omp.yield(%arg1 : !llvm.ptr)
+} copy {
+^bb0(%arg0: !llvm.ptr, %arg1: !llvm.ptr):
+ omp.yield(%arg1 : !llvm.ptr)
+} dealloc {
+^bb0(%arg0: !llvm.ptr):
+ omp.yield
+}
+
+omp.declare_reduction @red : i32
+init {
+^bb0(%arg0: i32):
+ llvm.call @red_callee() : () -> ()
+ omp.yield (%arg0 : i32)
+}
+combiner {
+^bb1(%arg0: i32, %arg1: i32):
+ omp.yield (%arg0 : i32)
+}
+cleanup {
+^bb0(%arg0: i32):
+ omp.yield
+}
+
+// CHECK: llvm.func {{.*}}@red_callee_nested2()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @red_callee_nested2() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@red_callee_nested()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @red_callee_nested() attributes {sym_visibility = "private"} {
+ llvm.call @red_callee_nested2() : () -> ()
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@red_callee()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @red_callee() attributes {sym_visibility = "private"} {
+ llvm.call @red_callee_nested() : () -> ()
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@priv_callee()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @priv_callee() attributes {sym_visibility = "private"} {
+ %0 = llvm.mlir.constant(1 : i64) : i64
+ %1 = llvm.alloca %0 x i32 : (i64) -> !llvm.ptr
+ omp.parallel reduction(@red %1 -> %arg0 : !llvm.ptr) {
+ omp.terminator
+ }
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@main_callee()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @main_callee() attributes {sym_visibility = "private"} {
+ %0 = llvm.mlir.constant(1 : i64) : i64
+ %1 = llvm.alloca %0 x i32 : (i64) -> !llvm.ptr
+ omp.parallel private(@priv %1 -> %arg0 : !llvm.ptr) {
+ omp.terminator
+ }
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@main()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @main() attributes {
+ omp.declare_target = #omp.declaretarget<
+ device_type = (nohost), capture_clause = (to), automap = false>} {
+ llvm.call @main_callee() : () -> ()
+ llvm.return
+}
+
+// -----
+
+// Non-declare target calling another non-declare target doesn't add any
+// attributes.
+
+// CHECK: llvm.func {{.*}}@callee()
+// CHECK-NOT: omp.declare_target
+// CHECK: llvm.return
+llvm.func @callee() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@main()
+// CHECK-NOT: omp.declare_target
+// CHECK: llvm.return
+llvm.func @main() {
+ llvm.call @callee() : () -> ()
+ llvm.return
+}
+
+// -----
+
+// declare_target calling another declare_target doesn't introduce changes.
+// If they aren't compatible, this is a user error.
+
+// CHECK: llvm.func {{.*}}@callee()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>
+llvm.func @callee() attributes {
+ sym_visibility = "private",
+ omp.declare_target = #omp.declaretarget<
+ device_type = (host), capture_clause = (to), automap = false>} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@main()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @main() attributes {
+ omp.declare_target = #omp.declaretarget<
+ device_type = (nohost), capture_clause = (to), automap = false>} {
+ llvm.call @callee() : () -> ()
+ llvm.return
+}
+
+// -----
+
+// Combining device_type(nohost) and device_type(host) results in
+// device_type(any) and it propagates to nested callees.
+
+// CHECK: llvm.func {{.*}}@callee_nested()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>
+llvm.func @callee_nested() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@callee()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>
+llvm.func @callee() attributes {sym_visibility = "private"} {
+ llvm.call @callee_nested() : () -> ()
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@fn_host()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>
+llvm.func @fn_host() attributes {
+ omp.declare_target = #omp.declaretarget<
+ device_type = (host), capture_clause = (to), automap = false>} {
+ llvm.call @callee() : () -> ()
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@fn_nohost()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @fn_nohost() attributes {
+ omp.declare_target = #omp.declaretarget<
+ device_type = (nohost), capture_clause = (to), automap = false>} {
+ llvm.call @callee() : () -> ()
+ llvm.return
+}
+
+// -----
+
+// Always use implicit device_type(any) for external and public functions.
+
+// CHECK: llvm.func {{.*}}@external_host()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>
+llvm.func @external_host()
+// CHECK: llvm.func {{.*}}@external_nohost()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>
+llvm.func @external_nohost()
+// CHECK: llvm.func {{.*}}@external_both()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>
+llvm.func @external_both()
+// CHECK: llvm.func {{.*}}@public_host()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>
+llvm.func @public_host() {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@public_nohost()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>
+llvm.func @public_nohost() {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@public_both()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (any), capture_clause = (to), automap = false>
+llvm.func @public_both() {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@fn_host()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>
+llvm.func @fn_host() attributes {
+ omp.declare_target = #omp.declaretarget<
+ device_type = (host), capture_clause = (to), automap = false>} {
+ llvm.call @external_host() : () -> ()
+ llvm.call @external_both() : () -> ()
+ llvm.call @public_host() : () -> ()
+ llvm.call @public_both() : () -> ()
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@fn_nohost()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @fn_nohost() attributes {
+ omp.declare_target = #omp.declaretarget<
+ device_type = (nohost), capture_clause = (to), automap = false>} {
+ llvm.call @external_nohost() : () -> ()
+ llvm.call @external_both() : () -> ()
+ llvm.call @public_nohost() : () -> ()
+ llvm.call @public_both() : () -> ()
+ llvm.return
+}
+
+// -----
+
+// omp.target private propagates device_type(nohost), unlike in_reduction.
+
+omp.private {type = firstprivate} @priv : !llvm.struct<(ptr)> init {
+^bb0(%arg0: !llvm.ptr, %arg1: !llvm.ptr):
+ llvm.call @priv_callee() : () -> ()
+ omp.yield(%arg1 : !llvm.ptr)
+} copy {
+^bb0(%arg0: !llvm.ptr, %arg1: !llvm.ptr):
+ omp.yield(%arg1 : !llvm.ptr)
+} dealloc {
+^bb0(%arg0: !llvm.ptr):
+ omp.yield
+}
+
+omp.declare_reduction @red : i32
+init {
+^bb0(%arg0: i32):
+ llvm.call @red_callee() : () -> ()
+ omp.yield (%arg0 : i32)
+}
+combiner {
+^bb1(%arg0: i32, %arg1: i32):
+ omp.yield (%arg0 : i32)
+}
+cleanup {
+^bb0(%arg0: i32):
+ omp.yield
+}
+
+// CHECK: llvm.func {{.*}}@red_callee()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (host), capture_clause = (to), automap = false>
+llvm.func @red_callee() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+// CHECK: llvm.func {{.*}}@priv_callee()
+// CHECK-SAME: omp.declare_target = #omp.declaretarget<device_type = (nohost), capture_clause = (to), automap = false>
+llvm.func @priv_callee() attributes {sym_visibility = "private"} {
+ llvm.return
+}
+
+llvm.func @main(%arg0 : !llvm.ptr) attributes {
+ omp.declare_target = #omp.declaretarget<
+ device_type = (host), capture_clause = (to), automap = false>} {
+ %0 = omp.map.info var_ptr(%arg0 : !llvm.ptr, i32) map_clauses(tofrom) capture(ByRef) -> !llvm.ptr
+ omp.target kernel_type(generic) in_reduction(@red %arg0 : !llvm.ptr)
+ map_entries(%0 -> %arg1 : !llvm.ptr)
+ private(@priv %arg0 -> %arg2 : !llvm.ptr) {
+ omp.terminator
+ }
+ llvm.return
+}
More information about the llvm-branch-commits
mailing list