<font size=2 face="sans-serif">Hi Jonas,</font><br><br><font size=2 face="sans-serif">The experiments are in this paper:</font><br><a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8287767"><font size=2 color=blue face="sans-serif">https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8287767</font></a><br><br><font size=2 face="sans-serif">In case you can't access the paper I'm
attaching it here.</font><br><br><font size=2 face="sans-serif">Thanks,</font><br><br><font size=2 face="sans-serif">--Doru</font><br><br><font size=2 face="sans-serif"><br></font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Jonas Hahnfeld <hahnjo@hahnjo.de></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">Gheorghe-Teod Bercea
<Gheorghe-Teod.Bercea@ibm.com></font><br><font size=1 color=#5f5f5f face="sans-serif">Cc:
</font><font size=1 face="sans-serif">Hal Finkel <hfinkel@anl.gov>,
Alexey Bataev <alexey.bataev@ibm.com>, openmp-dev@lists.llvm.org</font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">09/08/2018 05:10 AM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [Openmp-dev]
nested parallelism in libomptarget-nvptx</font><br><hr noshade><br><br><br><tt><font size=2>Hi Doru,<br><br>On 2018-09-07 22:03, Gheorghe-Teod Bercea wrote:<br>> Hi Hal,<br>> <br>> At least as far as we are aware, the number of use cases where the<br>> nested parallel scheme would be used is quite small. Most of the use<br>> cases of OpenMP on GPUs have a single level of parallelism which is<br>> typically SPMD-like to achieve as much performance as possible. That<br>> said there is some merit to having a nested parallelism scheme because<br>> when it is helpful it typically is very helpful.<br>> <br>> As a novelty point to ykt-clang I would suggest that whichever scheme<br>> (or schemes) we decide to use, they should be applied only at the<br>> request of the user. This is because we can do a better code gen job<br>> for more OpenMP patterns when using existing schemes (generic and<br>> SPMD) if we know at compile time if there will be no second level<br>> parallelism in use. This is due to some changes in implementation
in<br>> trunk compared to ykt-clang.<br><br>I agree: Even then we may be able to restructure the application to be
<br>more performant and portable without nested parallelism.<br><br>> Regarding which scheme to use there were two which were floated around<br>> based on discussions with users: (1) the current scheme in ykt-clang<br>> which enables the code in both inner and outer parallel loops to be<br>> executed in parallel and (2) a scheme where the outer loop code is<br>> executed by one thread and the innermost loop is executed by all<br>> threads (this was requested by users at one point, I assume this is<br>> still the case).<br>> <br>> Since ykt-clang only supports the fist scheme when we ran performance<br>> tests comparing nested parallelism against no nested parallelism we<br>> got anywhere from 4x slowdown to 32x speedup depending on the: ratio<br>> of outer:inner iterations, the work size in the innermost loop,<br>> reductions, atomics and memory coalescing. About 80% of the number
of<br>> cases we tried showed speed-ups with some showing significant<br>> speed-ups.<br>> I would very much be in favour of having at least this scheme<br>> supported since it looks like it could be useful.<br><br>Interesting. Are these experiments public? I'd be interested to see the
<br>codes that benefit from nested parallelism.<br>IIRC OpenACC doesn't have this feature, so I expect this to be corner <br>cases.<br><br>Regards,<br>Jonas<br><br></font></tt><br><br><BR>