[Openmp-commits] [openmp] r348758 - [OPENMP][NVPTX]Enable fast shuffles on 64bit values only if CUDA >= 9.

[Alexey Bataev via Openmp-commits]

Author: abataev
Date: Mon Dec 10 06:29:05 2018
New Revision: 348758

URL: http://llvm.org/viewvc/llvm-project?rev=348758&view=rev
Log:
[OPENMP][NVPTX]Enable fast shuffles on 64bit values only if CUDA >= 9.

Summary:
Shuffle on 64bit data is allowed only for CUDA >= 9.0. Also, fixed the
constant for the mask, need one extra L in the end.

Reviewers: gtbercea, kkwli0

Subscribers: guansong, caomhin, openmp-commits

Differential Revision: https://reviews.llvm.org/D55440

Modified:
    openmp/trunk/libomptarget/deviceRTLs/nvptx/src/reduction.cu

Modified: openmp/trunk/libomptarget/deviceRTLs/nvptx/src/reduction.cu
URL: http://llvm.org/viewvc/llvm-project/openmp/trunk/libomptarget/deviceRTLs/nvptx/src/reduction.cu?rev=348758&r1=348757&r2=348758&view=diff
==============================================================================
--- openmp/trunk/libomptarget/deviceRTLs/nvptx/src/reduction.cu (original)
+++ openmp/trunk/libomptarget/deviceRTLs/nvptx/src/reduction.cu Mon Dec 10 06:29:05 2018
@@ -76,7 +76,17 @@ EXTERN int32_t __kmpc_shuffle_int32(int3
 }
 
 EXTERN int64_t __kmpc_shuffle_int64(int64_t val, int16_t delta, int16_t size) {
-  return __SHFL_DOWN_SYNC(0xFFFFFFFFFFFFFFFFL, val, delta, size);
+#if defined(CUDART_VERSION) && CUDART_VERSION >= 9000
+  return __SHFL_DOWN_SYNC(0xFFFFFFFFFFFFFFFFLL, (long long)val, (unsigned)delta,
+                          (int)size);
+#else
+   int lo, hi;
+   asm volatile("mov.b64 {%0,%1}, %2;" : "=r"(lo), "=r"(hi) : "l"(val));
+   hi = __SHFL_DOWN_SYNC(0xFFFFFFFF, hi, delta, size);
+   lo = __SHFL_DOWN_SYNC(0xFFFFFFFF, lo, delta, size);
+   asm volatile("mov.b64 %0, {%1,%2};" : "=l"(val) : "r"(lo), "r"(hi));
+   return val;
+#endif
 }
 
 static INLINE void gpu_regular_warp_reduce(void *reduce_data,