[Openmp-commits] [PATCH] D47962: Introduce hierarchical scheduling

[Jonathan Peyton via Phabricator via Openmp-commits]

jlpeyton created this revision.
jlpeyton added reviewers: tlwilmar, AndreyChurbanov.
jlpeyton added a project: OpenMP.
Herald added a subscriber: mgorny.

This patch introduces the logic implementing hierarchical scheduling.
First and foremost, hierarchical scheduling is off by default
To enable, use `-DLIBOMP_USE_HIER_SCHED=On` during CMake's configure stage.
This work is based off if the IWOMP paper "Workstealing and Nested Parallelism in SMP Systems"

Hierarchical scheduling is the layering of OpenMP schedules for different layers
of the memory hierarchy.  One can have multiple layers between the threads and
the global iterations space.  The threads will go up the hierarchy to grab
iterations, using possibly a different schedule & chunk for each layer.

[ Global iteration space (0-999) ]

  (use static)

[    https://reviews.llvm.org/L1    |    https://reviews.llvm.org/L1    |   https://reviews.llvm.org/L1   |   https://reviews.llvm.org/L1   ]

  (use dynamic,1)

[ T0   T1  | T2    T3 | T4    T5 | T6   T7  ]

In the example shown above, there are 8 threads and 4 https://reviews.llvm.org/L1 caches begin targeted.
If the topology indicates that there are two threads per core, then two
consecutive threads will share the data of one https://reviews.llvm.org/L1 cache unit.  This example
would have the iteration space (0-999) split statically across the four https://reviews.llvm.org/L1
caches (so the first https://reviews.llvm.org/L1 would get (0-249), the second would get (250-499), etc).
Then the threads will use a dynamic,1 schedule to grab iterations from the https://reviews.llvm.org/L1
cache units.  There are currently four supported layers: https://reviews.llvm.org/L1, L2, L3, NUMA

`OMP_SCHEDULE` can now read a hierarchical schedule with this syntax:
`OMP_SCHEDULE='EXPERIMENTAL LAYER,SCHED[,CHUNK][:LAYER,SCHED[,CHUNK]...]:SCHED,CHUNK`
And of course `OMP_SCHEDULE` can still read the normal SCHED,CHUNK syntax from before

I've kept most of the hierarchical scheduling logic inside `kmp_dispatch_hier.h` to try to keep it separate from the rest of the code.


Repository:
  rOMP OpenMP

https://reviews.llvm.org/D47962

Files:
  runtime/CMakeLists.txt
  runtime/src/i18n/en_US.txt
  runtime/src/kmp.h
  runtime/src/kmp_affinity.cpp
  runtime/src/kmp_config.h.cmake
  runtime/src/kmp_dispatch.cpp
  runtime/src/kmp_dispatch.h
  runtime/src/kmp_dispatch_hier.h
  runtime/src/kmp_global.cpp
  runtime/src/kmp_runtime.cpp
  runtime/src/kmp_settings.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D47962.150578.patch
Type: text/x-patch
Size: 73294 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/openmp-commits/attachments/20180608/a95b7411/attachment-0001.bin>