[Openmp-commits] [openmp] 59a994e - [OpenMP][OMPT] thread_num determination for programs with explicit tasks

[Joachim Protze via Openmp-commits]

Author: @vladaindjic
Date: 2021-10-18T13:54:22+02:00
New Revision: 59a994e8daeef94976841a11bc92ca3a2b6a10b3

URL: https://github.com/llvm/llvm-project/commit/59a994e8daeef94976841a11bc92ca3a2b6a10b3
DIFF: https://github.com/llvm/llvm-project/commit/59a994e8daeef94976841a11bc92ca3a2b6a10b3.diff

LOG: [OpenMP][OMPT] thread_num determination for programs with explicit tasks

__ompt_get_task_info_internal is now able to determine the right value of the
“thread_num” argument during the execution of an explicit task.

During the execution of a while loop that iterates over the ancestor tasks
hierarchy, the “prev_team” variable was always set to “team” variable at the
beginning of each loop iteration.

Assume that the program contains a parallel region which encloses an explicit
task executed by the worker thread of the region. Also assume that the tool
inquires the “thread_num” of a worker thread for the implicit task that
corresponds to the region (task at “ancestor_level == 1”) and expects to
receive the value of “thread_num > 0”.
After the loop finishes, both “team” and “prev_team” variables are equal and
point to the team information of the parallel region.
The “thread_num” is set to “prev_team->t.t_master_tid”, that is equal to
“team->t.t_master_tid”. In this case, “team->t.t_master_tid” is 0, since
the master thread of the region is the initial master thread of the program.
This leads to a contradiction.

To prevent this, “prev_team” variable is set to “team” variable only at the
time when the loop that has already encountered the implicit task (“taskdata”
variable contains the information about an implicit task) continues iterating
over the implicit task’s ancestors, if any.

After the mentioned loop finishes, the “prev_team” variable might be equal to
NULL. This means that the task at requested “ancestor_level” belongs to the
innermost parallel region, so the “thread_num” will be determined by calling
the “__kmp_get_tid”.

To prove that this patch works, the test case “explicit_task_thread_num.c” is
provided.
It contains the example of the program explained earlier in the summary.

Differential Revision: https://reviews.llvm.org/D110473

Added: 
    openmp/runtime/test/ompt/tasks/explicit_task_thread_num.c

Modified: 
    openmp/runtime/src/ompt-specific.cpp

Removed: 
    


################################################################################
diff  --git a/openmp/runtime/src/ompt-specific.cpp b/openmp/runtime/src/ompt-specific.cpp
index c90edebc46a9c..f662dcdd1a005 100644
--- a/openmp/runtime/src/ompt-specific.cpp
+++ b/openmp/runtime/src/ompt-specific.cpp
@@ -369,7 +369,6 @@ int __ompt_get_task_info_internal(int ancestor_level, int *type,
 
     while (ancestor_level > 0) {
       // needed for thread_num
-      prev_team = team;
       prev_lwt = lwt;
       // next lightweight team (if any)
       if (lwt)
@@ -389,6 +388,7 @@ int __ompt_get_task_info_internal(int ancestor_level, int *type,
           taskdata = taskdata->td_parent;
           if (team == NULL)
             return 0;
+          prev_team = team;
           team = team->t.t_parent;
           if (taskdata) {
             next_lwt = LWT_FROM_TEAM(taskdata->td_team);
@@ -432,7 +432,16 @@ int __ompt_get_task_info_internal(int ancestor_level, int *type,
         *thread_num = __kmp_get_tid();
       else if (prev_lwt)
         *thread_num = 0;
-      else
+      else if (!prev_team) {
+        // The innermost parallel region contains at least one explicit task.
+        // The task at level > 0 is either an implicit task that
+        // corresponds to the mentioned region or one of the explicit tasks
+        // nested inside the same region. Note that the task isn't the
+        // innermost explicit tasks (because of condition level > 0).
+        // Since the task at this level still belongs to the innermost parallel
+        // region, thread_num is determined the same way as for level==0.
+        *thread_num = __kmp_get_tid();
+      } else
         *thread_num = prev_team->t.t_master_tid;
       //        *thread_num = team->t.t_master_tid;
     }

diff  --git a/openmp/runtime/test/ompt/tasks/explicit_task_thread_num.c b/openmp/runtime/test/ompt/tasks/explicit_task_thread_num.c
new file mode 100644
index 0000000000000..9f2d4abfbf747
--- /dev/null
+++ b/openmp/runtime/test/ompt/tasks/explicit_task_thread_num.c
@@ -0,0 +1,68 @@
+// RUN: %libomp-compile-and-run | FileCheck %s
+// REQUIRES: ompt
+
+#include "callback.h"
+#include <omp.h>
+
+
+__attribute__ ((noinline)) // workaround for bug in icc
+void print_task_info_at(int ancestor_level, int id)
+{
+#pragma omp critical
+  {
+    int task_type;
+    char buffer[2048];
+    ompt_data_t *parallel_data;
+    ompt_data_t *task_data;
+    int thread_num;
+    ompt_get_task_info(ancestor_level, &task_type, &task_data, NULL,
+                       &parallel_data, &thread_num);
+    format_task_type(task_type, buffer);
+    printf("%" PRIu64 ": ancestor_level=%d id=%d task_type=%s=%d "
+                      "parallel_id=%" PRIu64 " task_id=%" PRIu64
+                      " thread_num=%d\n",
+        ompt_get_thread_data()->value, ancestor_level, id, buffer,
+        task_type, parallel_data->value, task_data->value, thread_num);
+  }
+};
+
+int main()
+{
+
+#pragma omp parallel num_threads(2)
+  {
+
+    if (omp_get_thread_num() == 1) {
+      // To assert that task is executed by the worker thread,
+      // if(0) is used in order to ensure that the task is immediately
+      // executed after its creation.
+#pragma omp task if(0)
+      {
+        // thread_num should be equal to 1 for both explicit and implicit task
+        print_task_info_at(0, 1);
+        print_task_info_at(1, 0);
+      };
+    }
+  }
+
+  // Check if libomp supports the callbacks for this test.
+  // CHECK-NOT: {{^}}0: Could not register callback 'ompt_event_parallel_begin'
+  // CHECK-NOT: {{^}}0: Could not register callback 'ompt_callback_task_create'
+  // CHECK-NOT: {{^}}0: Could not register callback 'ompt_callback_implicit_task'
+
+  // CHECK: {{^}}0: NULL_POINTER=[[NULL:.*$]]
+  // CHECK: {{^}}[[MASTER_ID:[0-9]+]]: ompt_event_initial_task_begin
+
+  // parallel region used only to determine worker thread id
+  // CHECK: {{^}}[[MASTER_ID]]: ompt_event_parallel_begin
+  // CHECK-DAG: {{^}}[[MASTER_ID]]: ompt_event_implicit_task_begin
+  // CHECK-DAG: {{^}}[[WORKER_ID:[0-9]+]]: ompt_event_implicit_task_begin
+
+  // thread_num must be equal to 1 for both explicit and the implicit tasks
+  // CHECK: {{^}}[[WORKER_ID]]: ancestor_level=0 id=1 task_type=ompt_task_explicit
+  // CHECK-SAME: thread_num=1
+  // CHECK: {{^}}[[WORKER_ID]]: ancestor_level=1 id=0 task_type=ompt_task_implicit
+  // CHECK-SAME: thread_num=1
+
+  return 0;
+}