[Openmp-commits] [PATCH] D94241: Fix hierarchical barrier

[Terry Wilmarth via Phabricator via Openmp-commits]

tlwilmar created this revision.
tlwilmar added reviewers: AndreyChurbanov, jlpeyton, Nawrin.
tlwilmar added a project: OpenMP.
tlwilmar requested review of this revision.
Herald added a reviewer: jdoerfert.
Herald added a subscriber: sstefan1.

Hierarchical barrier is an experimental barrier algorithm that uses aspects of machine hierarchy to define the barrier tree structure.  This patch fixes offset calculation in hierarchical barrier. The offset is used to store info on a flag about sleeping threads waiting on a location stored in the flag. This commit also fixes a potential deadlock in  hierarchical barrier when using infinite blocktime by adjusting the offset value of leaf kids so that it matches the value of leaf state. It also adds testing of default barriers with infinite blocktime, and also tests hierarchical barrier algorithm with both default and infinite blocktime.

      

Patch by Terry Wilmarth and Nawrin Sultana.


Repository:
  rOMP OpenMP

https://reviews.llvm.org/D94241

Files:
  openmp/runtime/src/kmp_barrier.cpp
  openmp/runtime/test/barrier/omp_barrier.c


Index: openmp/runtime/test/barrier/omp_barrier.c
===================================================================
--- openmp/runtime/test/barrier/omp_barrier.c
+++ openmp/runtime/test/barrier/omp_barrier.c
@@ -1,4 +1,7 @@
 // RUN: %libomp-compile-and-run
+// RUN: %libomp-compile && env KMP_BLOCKTIME=infinite %libomp-run
+// RUN: %libomp-compile && env KMP_PLAIN_BARRIER_PATTERN='hierarchical,hierarchical' KMP_FORKJOIN_BARRIER_PATTERN='hierarchical,hierarchical' %libomp-run
+// RUN: %libomp-compile && env KMP_BLOCKTIME=infinite KMP_PLAIN_BARRIER_PATTERN='hierarchical,hierarchical' KMP_FORKJOIN_BARRIER_PATTERN='hierarchical,hierarchical' %libomp-run
 #include <stdio.h>
 #include "omp_testsuite.h"
 #include "omp_my_sleep.h"
Index: openmp/runtime/src/kmp_barrier.cpp
===================================================================
--- openmp/runtime/src/kmp_barrier.cpp
+++ openmp/runtime/src/kmp_barrier.cpp
@@ -828,8 +828,8 @@
           thr_bar->parent_tid = 0;
           thr_bar->my_level = d;
           break;
-        } else if ((rem = tid % thr_bar->skip_per_level[d + 1]) !=
-                   0) { // TODO: can we make this op faster?
+        } else if ((rem = tid % thr_bar->skip_per_level[d + 1]) != 0) {
+          // TODO: can we make the above op faster?
           // thread is not a subtree root at next level, so this is max
           thr_bar->parent_tid = tid - rem;
           thr_bar->my_level = d;
@@ -838,7 +838,9 @@
         ++d;
       }
     }
-    __kmp_type_convert(7 - (tid - thr_bar->parent_tid - 1), &(thr_bar->offset));
+    __kmp_type_convert(7 - ((tid - thr_bar->parent_tid) /
+                            (thr_bar->skip_per_level[thr_bar->my_level])),
+                       &(thr_bar->offset));
     thr_bar->old_tid = tid;
     thr_bar->wait_flag = KMP_BARRIER_NOT_WAITING;
     thr_bar->team = team;
@@ -1029,7 +1031,8 @@
     } else {
       // Leaf does special release on "offset" bits of parent's b_arrived flag
       thr_bar->b_arrived = team->t.t_bar[bt].b_arrived + KMP_BARRIER_STATE_BUMP;
-      kmp_flag_oncore flag(&thr_bar->parent_bar->b_arrived, thr_bar->offset);
+      kmp_flag_oncore flag(&thr_bar->parent_bar->b_arrived,
+                           thr_bar->offset + 1);
       flag.set_waiter(other_threads[thr_bar->parent_tid]);
       flag.release();
     }
@@ -1078,7 +1081,7 @@
       // Wait on my "offset" bits on parent's b_go flag
       thr_bar->wait_flag = KMP_BARRIER_PARENT_FLAG;
       kmp_flag_oncore flag(&thr_bar->parent_bar->b_go, KMP_BARRIER_STATE_BUMP,
-                           thr_bar->offset, bt,
+                           thr_bar->offset + 1, bt,
                            this_thr USE_ITT_BUILD_ARG(itt_sync_obj));
       flag.wait(this_thr, TRUE);
       if (thr_bar->wait_flag ==
@@ -1087,7 +1090,7 @@
               KMP_INIT_BARRIER_STATE); // Reset my b_go flag for next time
       } else { // Reset my bits on parent's b_go flag
         (RCAST(volatile char *,
-               &(thr_bar->parent_bar->b_go)))[thr_bar->offset] = 0;
+               &(thr_bar->parent_bar->b_go)))[thr_bar->offset + 1] = 0;
       }
     }
     thr_bar->wait_flag = KMP_BARRIER_NOT_WAITING;


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D94241.315163.patch
Type: text/x-patch
Size: 3180 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/openmp-commits/attachments/20210107/419968fe/attachment.bin>