<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=koi8-r">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0">> <span style="font-size: 11pt; font-family: Calibri, sans-serif, serif, "EmojiFont";">
We are working on OpenMP target offloading for GPUs in Flang, and adopting the same code generation strategy.</span></p>
<p style="margin-top:0;margin-bottom:0"><br>
<span style="font-size: 11pt; font-family: Calibri, sans-serif, serif, "EmojiFont";"></span></p>
<p style="margin-top:0;margin-bottom:0">Great to hear that that Flang is making progress on this front. I hope we can find ways to generalize/uncouple</p>
<p style="margin-top:0;margin-bottom:0">the OpenMP code generation in Clang so we do not need to re-implement it completely. That could also allow</p>
<p style="margin-top:0;margin-bottom:0">other language front-ends easier access to the OpenMP runtime for parallelization. Anyway, that is a different</p>
<p style="margin-top:0;margin-bottom:0">topic we should discuss separately.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"><span style="font-size: 11pt; font-family: Calibri, sans-serif, serif, "EmojiFont";"></span></p>
<p style="margin-top:0;margin-bottom:0"><span style="font-size: 11pt; font-family: Calibri, sans-serif, serif, "EmojiFont";">> The proposal is affecting us. It would be nice to know more details about the proposal. So we can prepare ourselves to adapt flang
(if everything goes on the way). </span></p>
<br>
<p></p>
<p style="margin-top:0;margin-bottom:0">As I mentioned before, this is an early design RFC, a lot of details need to be figured out.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">However, I do not think you will need to adapt much if you have (or reuse) a code generation similar to</p>
<p style="margin-top:0;margin-bottom:0">the SPMD code generation path in Clang. The goal of the first step is to make the code generation</p>
<p style="margin-top:0;margin-bottom:0">always look like that so in the best case you won't need to implement the non-SPMD code generation.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">> <span style="font-size: 11pt; font-family: Calibri, sans-serif, serif, "EmojiFont";">
Have you find and a solution for data sharing? How are you going to manage data sharing for
<span class="highlight" id="0.7222932657581529" name="searchHitInReadingPane">SPMD</span> and non-<span class="highlight" id="0.08241204901455113" name="searchHitInReadingPane">SPMD</span>?</span><br>
</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">Initially, we generate the same SPMD code as we do now. Data sharing for non-SPMD code is also not altered.</p>
<p style="margin-top:0;margin-bottom:0">The interesting question is how we can analyze non-SPMD code and <span>transform</span> it to SPMD code as easily as possible.</p>
<p style="margin-top:0;margin-bottom:0">That might require us to change the "encoding" later down the road but for now, we won't change anything in this regard.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Guray Ozen <gozen@nvidia.com><br>
<b>Sent:</b> Wednesday, January 23, 2019 3:14:57 AM<br>
<b>To:</b> Doerfert, Johannes Rudolf; Alexey Bataev<br>
<b>Cc:</b> llvm-dev; openmp-dev@lists.llvm.org<br>
<b>Subject:</b> RE: [RFC] Late (OpenMP) GPU code "SPMD-zation"</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
@font-face
{font-family:Consolas}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif}
a:link, span.x_MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.x_MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
pre
{margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New"}
p.x_msonormal0, li.x_msonormal0, div.x_msonormal0
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif}
span.x_HTMLPreformattedChar
{font-family:Consolas}
span.x_EmailStyle22
{font-family:"Calibri",sans-serif}
.x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-US" link="blue" vlink="purple">
<div class="x_WordSection1">
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif">We are working on OpenMP target offloading for GPUs in Flang, and adopting the same code generation strategy. The proposal is affecting us. It would be nice to know more
details about the proposal. So we can prepare ourselves to adapt flang (if everything goes on the way).
</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif">Have you find and a solution for data sharing? How are you going to manage data sharing for SPMD and non-SPMD?</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif"> </span></p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif"> cfe-dev <cfe-dev-bounces@lists.llvm.org>
<b>On Behalf Of </b>Doerfert, Johannes Rudolf via cfe-dev<br>
<b>Sent:</b> Wednesday, January 23, 2019 12:50 AM<br>
<b>To:</b> Alexey Bataev <a.bataev@outlook.com><br>
<b>Cc:</b> llvm-dev <llvm-dev@lists.llvm.org>; cfe-dev@lists.llvm.org; openmp-dev@lists.llvm.org<br>
<b>Subject:</b> Re: [cfe-dev] [RFC] Late (OpenMP) GPU code "SPMD-zation"</span></p>
</div>
</div>
<p class="x_MsoNormal"> </p>
<div id="x_divtagdefaultwrapper">
<p><span style="font-family:"Calibri",sans-serif; color:black">After an IRC discussion, I think Alexey and I are pretty much in agreement (on the general feasibility at least).</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">I try to sketch the proposed idea again below, as the initial RFC was simply not descriptive enough.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">After that, I shortly summarize how I see these changes being developed and committed so that we</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> - never have any regressions,</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> - can make an educated decision before removing any existing code.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">What we want to do:</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">The intermediate goal is that the code generated by clang for the SPMD and non-SPMD (earlier denoted as "guarded") case</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">is conceptually/structurally very similar. The current non-SPMD code is, however, a state machine generated into the user code
</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">module. This state machine is very hard to analyze and optimize. If the code would look as the SPMD code but *behave the same</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">way it does now*, we could "easily" switch from non-SPMD to SPMD version after a (late) analysis determined legality. To make</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">the code look the same but behave differently, we propose to hide the semantic difference in the runtime library calls. That is,
</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">the runtime calls emitted in the two modes are (slightly) different, or there is a flag which indicates the (initial) mode. If that mode</span></p>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">is SPMD, the runtime behavior does not change compared to the way it is now. If that mode is non-SPMD, the runtime would
</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">separate the master and worker threads, as we do it now in the user code module, and keep the workers in an internal state machine</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">waiting for the master to provide them with work. Only the master would return from the runtime call and the mechanism</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">to distribute work to the worker threads would (for now) stay the same.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">Preliminary implementation (and integration) steps:</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">1) Design and implement the necessary runtime extensions and determine feasibility.
</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">2) Allow to Clang codegen to use the new runtime extensions if explicitly chosen by the user.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">2b) Performance comparison unoptimized new code path vs. original code path on test cases and real use cases.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">3) Implement the middle-end pass to analyze and optimize the code using the runtime extensions.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">3b) Performance comparison optimized new code path vs. original code path on real use cases.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">4) If no conceptual problem was found and 2b)/3b) determined that the new code path is superior, switch to the</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> new code path by default.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">5) If no regressions/complaints are reported after a grace period, remove the old code path from the clang front-end.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">Again, this is an early design RFC for which I welcome any feedback!</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">Thanks,</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> Johannes</span></p>
<div id="x_divtagdefaultwrapper">
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div class="x_MsoNormal" align="center" style="text-align:center"><span style="font-family:"Calibri",sans-serif; color:black">
<hr size="2" width="98%" align="center">
</span></div>
<div id="x_divRplyFwdMsg">
<p class="x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black"> Doerfert, Johannes Rudolf<br>
<b>Sent:</b> Tuesday, January 22, 2019 1:50:51 PM<br>
<b>To:</b> Alexey Bataev<br>
<b>Cc:</b> <a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a>; <a href="mailto:openmp-dev@lists.llvm.org">
openmp-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [RFC] Late (OpenMP) GPU code "SPMD-zation"</span><span style="font-family:"Calibri",sans-serif; color:black">
</span></p>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
</div>
<div>
<div id="x_x_divtagdefaultwrapper">
<p><span style="font-family:"Calibri",sans-serif; color:black">What do you refer to with: "No, we don't". </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">Again, I do not propose to remove the SPMD "detection" in Clang. We will still identify SPMD mode based on the syntactic criteria we have now.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black">The Clang analysis is also not affected. Thus, we will globalize/localize the same variables as we do now. I don't see why this should be any different.</span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div class="x_MsoNormal" align="center" style="text-align:center"><span style="font-family:"Calibri",sans-serif; color:black">
<hr size="2" width="98%" align="center">
</span></div>
<div id="x_x_divRplyFwdMsg">
<p class="x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black"> llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org">llvm-dev-bounces@lists.llvm.org</a>>
on behalf of Alexey Bataev via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>><br>
<b>Sent:</b> Tuesday, January 22, 2019 1:46:39 PM<br>
<b>To:</b> Doerfert, Johannes Rudolf<br>
<b>Cc:</b> llvm-dev; <a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a>;
<a href="mailto:openmp-dev@lists.llvm.org">openmp-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] [RFC] Late (OpenMP) GPU code "SPMD-zation"</span><span style="font-family:"Calibri",sans-serif; color:black">
</span></p>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
</div>
<div>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">No, we don't. We need to perform the different kind of the analysis for SPMD mode constructs and Non-SPMD.</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">For SPMD mode we need to globalize only reduction/lastprivate variables. For Non-SPMD mode, we need to globalize all the private/local variables, that may escape their declaration
context in the construct.</span></p>
<pre style="background:white"><span style="color:black">-------------</span></pre>
<pre style="background:white"><span style="color:black">Best regards,</span></pre>
<pre style="background:white"><span style="color:black">Alexey Bataev</span></pre>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">22.01.2019 14:29, Doerfert, Johannes Rudolf пишет:</span></p>
</div>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<div id="x_x_x_divtagdefaultwrapper">
<p class="x_MsoNormal" style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">We would still know that. We can do exactly the same reasoning as we do now.
</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">I think the important question is, how different is the code generated for either mode and can we hide (most of) the differences in the runtime.</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">If I understand you correctly, you say the data sharing code looks very different and the differences cannot be hidden, correct?</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">It would be helpful for me to understand your point if you could give me a piece of OpenMP for which the data sharing in SPMD mode and "guarded"</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">mode are as different as possible. I can compile it in both modes myself so high-level OpenMP is fine (I will disable SPMD mode manually in the source if necessary).</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">Thanks,</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black"> Johannes</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<p class="x_MsoNormal" style="margin-bottom:12.0pt; background:white"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<div>
<div class="x_MsoNormal" align="center" style="text-align:center; background:white">
<span style="font-family:"Calibri",sans-serif; color:black">
<hr size="2" width="98%" align="center">
</span></div>
<div id="x_x_x_divRplyFwdMsg">
<p class="x_MsoNormal" style="background:white"><b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black"> llvm-dev
<a href="mailto:llvm-dev-bounces@lists.llvm.org"><llvm-dev-bounces@lists.llvm.org></a> on behalf of Alexey Bataev via llvm-dev
<a href="mailto:llvm-dev@lists.llvm.org"><llvm-dev@lists.llvm.org></a><br>
<b>Sent:</b> Tuesday, January 22, 2019 13:10<br>
<b>To:</b> Doerfert, Johannes Rudolf<br>
<b>Cc:</b> Alexey Bataev; LLVM-Dev; Arpith Chacko Jacob; <a href="mailto:openmp-dev@lists.llvm.org">
openmp-dev@lists.llvm.org</a>; <a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] [RFC] Late (OpenMP) GPU code "SPMD-zation"</span><span style="font-family:"Calibri",sans-serif; color:black">
</span></p>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
</div>
<div>
<div>
<p class="x_MsoNormal" style="margin-bottom:12.0pt; background:white"><span style="font-family:"Calibri",sans-serif; color:black">But we need to know the execution mode, SPMD or "guarded"</span></p>
</div>
<pre style="background:white"><span style="color:black">-------------</span></pre>
<pre style="background:white"><span style="color:black">Best regards,</span></pre>
<pre style="background:white"><span style="color:black">Alexey Bataev</span></pre>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">22.01.2019 13:54, Doerfert, Johannes Rudolf пишет:</span></p>
</div>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<div>
<p class="x_MsoNormal" style="margin-bottom:12.0pt; background:white"><span style="font-size:11.0pt; font-family:"Arial",sans-serif; color:black">We could still do that in clang, couldn't we?</span></p>
</div>
<div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.0pt; font-family:"Arial",sans-serif; color:black">Get
<a href="https://aka.ms/ghei36">Outlook for Android</a></span></p>
</div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.0pt; font-family:"Arial",sans-serif; color:black"> </span></p>
</div>
<div class="x_MsoNormal" align="center" style="text-align:center; background:white">
<span style="font-family:"Calibri",sans-serif; color:black">
<hr size="2" width="98%" align="center">
</span></div>
<div id="x_x_x_x_divRplyFwdMsg">
<p class="x_MsoNormal" style="background:white"><b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black"> Alexey Bataev
<a href="mailto:a.bataev@outlook.com"><a.bataev@outlook.com></a><br>
<b>Sent:</b> Tuesday, January 22, 2019 12:52:42 PM<br>
<b>To:</b> Doerfert, Johannes Rudolf; <a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a><br>
<b>Cc:</b> <a href="mailto:openmp-dev@lists.llvm.org">openmp-dev@lists.llvm.org</a>; LLVM-Dev; Finkel, Hal J.; Alexey Bataev; Arpith Chacko Jacob<br>
<b>Subject:</b> Re: [RFC] Late (OpenMP) GPU code "SPMD-zation"</span><span style="font-family:"Calibri",sans-serif; color:black">
</span></p>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
</div>
<div>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">The globalization for the local variables, for example. It must be implemented in the compiler to get the good performance, not in the runtime.</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<pre style="background:white"><span style="color:black">-------------</span></pre>
<pre style="background:white"><span style="color:black">Best regards,</span></pre>
<pre style="background:white"><span style="color:black">Alexey Bataev</span></pre>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">22.01.2019 13:43, Doerfert, Johannes Rudolf пишет:</span></p>
</div>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<div>
<p class="x_MsoNormal" style="margin-bottom:12.0pt; background:white"><span style="font-size:11.0pt; font-family:"Arial",sans-serif; color:black">Could you elaborate on what you refer to wrt data sharing. What do we currently do in the clang code generation
that we could not effectively implement in the runtime, potentially with support of an llvm pass.</span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.0pt; font-family:"Arial",sans-serif; color:black">Thanks,</span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin-bottom:12.0pt; background:white"><span style="font-size:11.0pt; font-family:"Arial",sans-serif; color:black"> James</span></p>
</div>
<div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.0pt; font-family:"Arial",sans-serif; color:black">Get
<a href="https://aka.ms/ghei36">Outlook for Android</a></span></p>
</div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.0pt; font-family:"Arial",sans-serif; color:black"> </span></p>
</div>
<div class="x_MsoNormal" align="center" style="text-align:center; background:white">
<span style="font-family:"Calibri",sans-serif; color:black">
<hr size="2" width="98%" align="center">
</span></div>
<div id="x_x_x_x_divRplyFwdMsg">
<p class="x_MsoNormal" style="background:white"><b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black"> Alexey Bataev
<a href="mailto:a.bataev@outlook.com"><a.bataev@outlook.com></a><br>
<b>Sent:</b> Tuesday, January 22, 2019 12:34:01 PM<br>
<b>To:</b> Doerfert, Johannes Rudolf; <a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a><br>
<b>Cc:</b> <a href="mailto:openmp-dev@lists.llvm.org">openmp-dev@lists.llvm.org</a>; LLVM-Dev; Finkel, Hal J.; Alexey Bataev; Arpith Chacko Jacob<br>
<b>Subject:</b> Re: [RFC] Late (OpenMP) GPU code "SPMD-zation"</span><span style="font-family:"Calibri",sans-serif; color:black">
</span></p>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
</div>
<div>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<pre style="background:white"><span style="color:black">-------------</span></pre>
<pre style="background:white"><span style="color:black">Best regards,</span></pre>
<pre style="background:white"><span style="color:black">Alexey Bataev</span></pre>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">22.01.2019 13:17, Doerfert, Johannes Rudolf пишет:</span></p>
</div>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<pre style="background:white"><span style="color:black">Where we are</span></pre>
<pre style="background:white"><span style="color:black">------------</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">Currently, when we generate OpenMP target offloading code for GPUs, we</span></pre>
<pre style="background:white"><span style="color:black">use sufficient syntactic criteria to decide between two execution modes:</span></pre>
<pre style="background:white"><span style="color:black"> 1) SPMD -- All target threads (in an OpenMP team) run all the code.</span></pre>
<pre style="background:white"><span style="color:black"> 2) "Guarded" -- The master thread (of an OpenMP team) runs the user</span></pre>
<pre style="background:white"><span style="color:black"> code. If an OpenMP distribute region is encountered, thus</span></pre>
<pre style="background:white"><span style="color:black"> if all threads (in the OpenMP team) are supposed to</span></pre>
<pre style="background:white"><span style="color:black"> execute the region, the master wakes up the idling</span></pre>
<pre style="background:white"><span style="color:black"> worker threads and points them to the correct piece of</span></pre>
<pre style="background:white"><span style="color:black"> code for distributed execution.</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">For a variety of reasons we (generally) prefer the first execution mode.</span></pre>
<pre style="background:white"><span style="color:black">However, depending on the code, that might not be valid, or we might</span></pre>
<pre style="background:white"><span style="color:black">just not know if it is in the Clang code generation phase.</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">The implementation of the "guarded" execution mode follows roughly the</span></pre>
<pre style="background:white"><span style="color:black">state machine description in [1], though the implementation is different</span></pre>
<pre style="background:white"><span style="color:black">(more general) nowadays.</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">What we want</span></pre>
<pre style="background:white"><span style="color:black">------------</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">Increase the amount of code executed in SPMD mode and the use of</span></pre>
<pre style="background:white"><span style="color:black">lightweight "guarding" schemes where appropriate.</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">How we get (could) there</span></pre>
<pre style="background:white"><span style="color:black">------------------------</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">We propose the following two modifications in order:</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black"> 1) Move the state machine logic into the OpenMP runtime library. That</span></pre>
<pre style="background:white"><span style="color:black"> means in SPMD mode all device threads will start the execution of</span></pre>
<pre style="background:white"><span style="color:black"> the user code, thus emerge from the runtime, while in guarded mode</span></pre>
<pre style="background:white"><span style="color:black"> only the master will escape the runtime and the other threads will</span></pre>
<pre style="background:white"><span style="color:black"> idle in their state machine code that is now just "hidden".</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black"> Why:</span></pre>
<pre style="background:white"><span style="color:black"> - The state machine code cannot be (reasonably) optimized anyway,</span></pre>
<pre style="background:white"><span style="color:black"> moving it into the library shouldn't hurt runtime but might even</span></pre>
<pre style="background:white"><span style="color:black"> improve compile time a little bit.</span></pre>
<pre style="background:white"><span style="color:black"> - The change should also simplify the Clang code generation as we</span></pre>
<pre style="background:white"><span style="color:black"> would generate structurally the same code for both execution modes</span></pre>
<pre style="background:white"><span style="color:black"> but only the runtime library calls, or their arguments, would</span></pre>
<pre style="background:white"><span style="color:black"> differ between them.</span></pre>
<pre style="background:white"><span style="color:black"> - The reason we should not "just start in SPMD mode" and "repair"</span></pre>
<pre style="background:white"><span style="color:black"> it later is simple, this way we always have semantically correct</span></pre>
<pre style="background:white"><span style="color:black"> and executable code.</span></pre>
<pre style="background:white"><span style="color:black"> - Finally, and most importantly, there is now only little</span></pre>
<pre style="background:white"><span style="color:black"> difference (see above) between the two modes in the code</span></pre>
<pre style="background:white"><span style="color:black"> generated by clang. If we later analyze the code trying to decide</span></pre>
<pre style="background:white"><span style="color:black"> if we can use SPMD mode instead of guarded mode the analysis and</span></pre>
<pre style="background:white"><span style="color:black"> transformation becomes much simpler.</span></pre>
</blockquote>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">The last item is wrong, unfortunately. A lot of things in the codegen depend on the execution mode, e.g. correct support of the data-sharing. Of course, we can try to generalize
the codegen and rely completely on the runtime, but the performance is going to be very poor.</span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif; color:black">We still need static analysis in the compiler. I agree, that it is better to move this analysis to the backend, at least after the inlining, but at the moment it is not
possible. We need the support for the late outlining, which will allow to implement better detection of the SPMD constructs + improve performance.</span></p>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<pre style="background:white"><span style="color:black"> 2) Implement a middle-end LLVM-IR pass that detects the guarded mode,</span></pre>
<pre style="background:white"><span style="color:black"> e.g., through the runtime library calls used, and that tries to</span></pre>
<pre style="background:white"><span style="color:black"> convert it into the SPMD mode potentially by introducing lightweight</span></pre>
<pre style="background:white"><span style="color:black"> guards in the process.</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black"> Why:</span></pre>
<pre style="background:white"><span style="color:black"> - After the inliner, and the canonicalizations, we have a clearer</span></pre>
<pre style="background:white"><span style="color:black"> picture of the code that is actually executed in the target</span></pre>
<pre style="background:white"><span style="color:black"> region and all the side effects it contains. Thus, we can make an</span></pre>
<pre style="background:white"><span style="color:black"> educated decision on the required amount of guards that prevent</span></pre>
<pre style="background:white"><span style="color:black"> unwanted side effects from happening after a move to SPMD mode.</span></pre>
<pre style="background:white"><span style="color:black"> - At this point we can more easily introduce different schemes to</span></pre>
<pre style="background:white"><span style="color:black"> avoid side effects by threads that were not supposed to run. We</span></pre>
<pre style="background:white"><span style="color:black"> can decide if a state machine is needed, conditionals should be</span></pre>
<pre style="background:white"><span style="color:black"> employed, masked instructions are appropriate, or "dummy" local</span></pre>
<pre style="background:white"><span style="color:black"> storage can be used to hide the side effect from the outside</span></pre>
<pre style="background:white"><span style="color:black"> world.</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">None of this was implemented yet but we plan to start in the immediate</span></pre>
<pre style="background:white"><span style="color:black">future. Any comments, ideas, criticism is welcome!</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">Cheers,</span></pre>
<pre style="background:white"><span style="color:black"> Johannes</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">P.S. [2-4] Provide further information on implementation and features.</span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black">[1] <a href="https://ieeexplore.ieee.org/document/7069297">https://ieeexplore.ieee.org/document/7069297</a></span></pre>
<pre style="background:white"><span style="color:black">[2] <a href="https://dl.acm.org/citation.cfm?id=2833161">https://dl.acm.org/citation.cfm?id=2833161</a></span></pre>
<pre style="background:white"><span style="color:black">[3] <a href="https://dl.acm.org/citation.cfm?id=3018870">https://dl.acm.org/citation.cfm?id=3018870</a></span></pre>
<pre style="background:white"><span style="color:black">[4] <a href="https://dl.acm.org/citation.cfm?id=3148189">https://dl.acm.org/citation.cfm?id=3148189</a></span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
<pre style="background:white"><span style="color:black"> </span></pre>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div>
<hr>
</div>
<div>This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message. </div>
<div>
<hr>
</div>
</div>
</body>
</html>