[Openmp-commits] [PATCH] D108488: [OpenMP] [test] Fix intermittent omp_get_wtime.c test failures

[Jonathan Peyton via Phabricator via Openmp-commits]

jlpeyton created this revision.
jlpeyton added reviewers: ronlieb, JonChesterfield, AndreyChurbanov.
jlpeyton added a project: OpenMP.
Herald added subscribers: guansong, yaxunl.
jlpeyton requested review of this revision.
Herald added a reviewer: jdoerfert.
Herald added a subscriber: sstefan1.

The omp_get_wtime.c test fails intermittently if the recorded times are off by too much which can happen when many tests are run in parallel.
Instead of failing if one timing is a little off, take average of 100 timings minus the 10 worst. Put the threshold at a generous 33%.

Brought to attention by Ron Lieberman (Thanks!)


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D108488

Files:
  openmp/runtime/test/api/omp_get_wtime.c


Index: openmp/runtime/test/api/omp_get_wtime.c
===================================================================
--- openmp/runtime/test/api/omp_get_wtime.c
+++ openmp/runtime/test/api/omp_get_wtime.c
@@ -4,30 +4,73 @@
 #include "omp_testsuite.h"
 #include "omp_my_sleep.h"
 
-int test_omp_get_wtime()
-{
+#define NTIMES 100
+
+// This is the error % threshold. Be generous with the error threshold since
+// this test may be run in parallel with many other tests it may throw off the
+// sleep timing.
+#define THRESHOLD 33.0
+
+double test_omp_get_wtime(double desired_wait_time) {
   double start;
   double end;
-  double measured_time;
-  double wait_time = 0.2;
   start = 0;
   end = 0;
   start = omp_get_wtime();
-  my_sleep (wait_time);
+  my_sleep(desired_wait_time);
   end = omp_get_wtime();
-  measured_time = end-start;
-  return ((measured_time > 0.97 * wait_time) && (measured_time < 1.03 * wait_time)) ;
+  return end - start;
 }
 
-int main()
-{
-  int i;
-  int num_failed=0;
+int compare_times(const void *lhs, const void *rhs) {
+  const double *a = (const double *)lhs;
+  const double *b = (const double *)rhs;
+  return *a - *b;
+}
+
+int main() {
+  int i, final_count;
+  double percent_off;
+  double *begin, *end, *ptr;
+  double wait_time = 0.01;
+  double average = 0.0;
+  double n = 0.0;
+  double *times = (double *)malloc(sizeof(double) * NTIMES);
+
+  // Get each timing
+  for (i = 0; i < NTIMES; i++) {
+    times[i] = test_omp_get_wtime(wait_time);
+  }
+
+  // Remove approx the "worst" tenth of the timings
+  qsort(times, NTIMES, sizeof(double), compare_times);
+  begin = times;
+  end = times + NTIMES;
+  for (i = 0; i < NTIMES / 10; ++i) {
+    if (i % 2 == 0)
+      begin++;
+    else
+      end--;
+  }
+
+  // Get the average of the remaining timings
+  for (ptr = begin, final_count = 0; ptr != end; ++ptr, ++final_count)
+    average += times[i];
+  average /= (double)final_count;
+  free(times);
+
+  // Calculate the percent off of desired wait time
+  percent_off = (average - wait_time) / wait_time * 100.0;
+  // Should always be positive, but just in case
+  if (percent_off < 0)
+    percent_off = -percent_off;
 
-  for(i = 0; i < REPETITIONS; i++) {
-    if(!test_omp_get_wtime()) {
-      num_failed++;
-    }
+  if (percent_off > (double)THRESHOLD) {
+    fprintf(stderr, "error: average of %d runs (%lf) is of by %lf%%\n", NTIMES,
+            average, percent_off);
+    return EXIT_FAILURE;
   }
-  return num_failed;
+  printf("pass: average of %d runs (%lf) is only off by %lf%%\n", NTIMES,
+         average, percent_off);
+  return EXIT_SUCCESS;
 }


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D108488.367891.patch
Type: text/x-patch
Size: 2622 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/openmp-commits/attachments/20210820/656c7298/attachment.bin>