[Openmp-commits] [PATCH] D81054: [OpenMP] Introduce target memory manager

[Shilei Tian via Phabricator via Openmp-commits]

tianshilei1992 created this revision.
Herald added subscribers: openmp-commits, sstefan1, guansong, yaxunl, mgorny.
Herald added a reviewer: jdoerfert.
Herald added a project: OpenMP.
tianshilei1992 updated this revision to Diff 268056.
tianshilei1992 added a comment.

Updated function names to conform with LLVM code standards


Target memory manager is introduced in this patch which aims to manage target
memory such that they will not be freed immediately when they are not used
because the overhead of memory allocation and free is very large. For CUDA
device, cuMemFree even blocks the context switch on device which affects
concurrent kernel execution.

The memory manager can be taken as a memory pool. It divides the pool into
multiple buckets according to the size such that memory allocation/free
distributed to different buckets will not affect each other.

In this version, we use the exact-equality policy to find a free buffer. This
is an open question: will best-fit work better here? IMO, best-fit is not good
for target memory management because computation on GPU usually requires GBs of
data. Best-fit might lead to a serious waste. For example, there is a free
buffer of size 1960MB, and now we need a buffer of size 1200MB. If best-fit,
the free buffer will be returned, leading to a 760MB waste.

The allocation will happen when there is no free memory left, and the memory
free on device will take place in the following two cases:

1. The program ends. Obviously. However, there is a little problem that plugin

library is destroyed before the memory manager is destroyed, leading to a fact
that the call to target plugin will not succeed.

2. Device is out of memory when we request a new memory. The manager will walk

through all free buffers from the bucket with largest base size, pick up one
buffer, free it, and try to allocate immediately. If it succeeds, it will
return right away rather than freeing all buffers in free list.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D81054

Files:
  openmp/libomptarget/src/CMakeLists.txt
  openmp/libomptarget/src/device.cpp
  openmp/libomptarget/src/memory.cpp
  openmp/libomptarget/src/memory.h
  openmp/libomptarget/src/omptarget.cpp
  openmp/libomptarget/src/rtl.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D81054.268056.patch
Type: text/x-patch
Size: 13109 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/openmp-commits/attachments/20200603/3f3ff45a/attachment-0001.bin>