[Openmp-commits] [PATCH] D77412: [OpenMP] Introduce stream pool to make sure the correctness of device synchronization

[Shilei Tian via Phabricator via Openmp-commits]

tianshilei1992 created this revision.
tianshilei1992 added a reviewer: jdoerfert.
tianshilei1992 added a project: OpenMP.
Herald added subscribers: openmp-commits, jfb, guansong, yaxunl.
tianshilei1992 added a parent revision: D77005: [OpenMP] Optimized stream selection by scheduling data mapping for the same target region into a same stream.
tianshilei1992 edited the summary of this revision.

In previous patch <https://reviews.llvm.org/D77005>, in order to optimize performance, we only synchronize once
for each target region. The syncrhonization is via stream synchronization.
However, in the extreme situation, the performce might be bad. Consider the
following case: There is a task that requires transferring huge amount of data
(call many times of data transferring function). It is scheduled to the first
stream. And then we have 255 very light tasks scheduled to the remaining 255
streams (by default we have 256 streams). They can be finished before we do
synchronization at the end of the first task. Next, we get another very huge
task. It will be scheduled again to the first stream. Now the first task
finishes its kernel launch and call stream synchronization. Right now, the
stream already contains two kernels, and the synchronization will wait until the
two kernels finish instead of just the first one for the first task.

In this patch, we introduce stream pool. After each synchronization, the stream
will be returned back to the pool to make sure that for each synchronization,
only expected operations are waited.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D77412

Files:
  openmp/libomptarget/plugins/cuda/src/rtl.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D77412.254836.patch
Type: text/x-patch
Size: 7077 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/openmp-commits/attachments/20200403/3ca4f3b4/attachment-0001.bin>