<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-size: 12pt; color: rgb(0, 0, 0);">
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px; font-size: 12pt; background-color: rgb(255, 255, 255);">
<span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">Hi all,<br>
</span></div>
<div style="margin: 0px; font-size: 12pt; background-color: rgb(255, 255, 255);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; margin: 0px; background-color: rgb(255, 255, 255); display: inline !important;"><br>
</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px; background-color: rgb(255, 255, 255); display: inline !important;">We gave a presentation [1] a few months back regarding the OpenMP design for F18 during the Flang/F18 technical
call, also sent a summary mail</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px; background-color: rgb(255, 255, 255); display: inline !important;"> [2] </span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px; background-color: rgb(255, 255, 255); display: inline !important;">and
a set of walkthroughs [3] to the mailing list. We received some feedback and have incorporated that into the design. </span><font face="Calibri, Arial, Helvetica, sans-serif" style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif;"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">A
summary of the design and status of the OpenMP implementation in F18 from lowering of the parse tree to LLVM IR generation is presented in this mail. (Note that Semantic and Structural checks are not covered. Refer to Gary's biweekly mail for the status.)<br>
</span><br>
<span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">The proposed design can be seen in slide 10 of the presentation [1]. </span></font><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><font face="Calibri, Helvetica, sans-serif"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">The
design summary is as follows.</span><br>
<span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">i) Uses the following two components.</span></font><font face="Calibri, Arial, Helvetica, sans-serif"><br>
</font></span><font face="Segoe UI, Segoe UI Web (West European), Segoe UI, -apple-system, system-ui, Roboto, Helvetica Neue, sans-serif"></font><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><font face="Calibri, Helvetica, sans-serif"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">a)
MLIR [4]: Necessary since MLIR has already been chosen as the framework for building the Fortran IR (FIR) for Flang. By using MLIR for OpenMP we have a common representation for OpenMP and Fortran constructs in the MLIR framework and thereby take advantage
of optimisations and avoid black boxes.</span></font><font face="Calibri, Arial, Helvetica, sans-serif"><br>
</font></span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><font face="Calibri, Helvetica, sans-serif"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">b)
OpenMP IRBuilder [5]: For sharing of OpenMP codegen with Clang.</span><br>
</font></span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><font face="Calibri, Helvetica, sans-serif"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"><br>
ii) Current and Proposed Flow</span><br>
</font></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">a) The current sequential code flow in Flang (Slide 5) of the presentation [1] can be summarised as follows,</span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><br>
</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">[Fortran code] -> Parser -> [AST] -> Lowering -> [FIR MLIR] -> Conversion -> [LLVM MLIR] -> Translation -> [LLVM IR]</span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><br>
</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">b) The modified flow with OpenMP (Slide 10) will have lowering of the AST to a mix of FIR and OpenMP dialects. These are then </span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">optimised
and finally converted to mix of OpenMP and LLVM MLIR dialects. The mix is translated to LLVM IR using the </span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">existing translation library for LLVM MLIR and the OpenMP IRBuilder currently
under construction.</span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><br>
</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">[Fortran code] -> Parser -> [AST] -> Lowering -> [FIR + OpenMP MLIR] -> Conversion -> [LLVM + OpenMP MLIR] -> Translation (Use OpenMP IRBuilder) -> [LLVM IR]<br>
c) The MLIR infrastructure provides a lot of optimisation passes for loops. It is desirable that we take advantage of some of these. But the LLVM infrastructure also provides several optimisations. So there exist some questions regarding where the optimisations
should be carried out. We will decide on which framework to choose only after some experimentation. If we decide that the OpenMP construct (for e.g. collapse) can be handled fully in MLIR and that is the best place to do it (based on experiments) then we will
not use the OpenMP IRBuilder for these constructs.<br>
<br>
iii) OpenMP MLIR dialect<br>
</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">The proposed plan involves writing an MLIR dialect for OpenMP.<span> </span></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">Operations of the dialect
will be a mix of fine and coarse-grained. e.g. Coarse: omp.parallel, omp.target, Fine: omp.flush. The operations in MLIR can have regions, hence there is no need for outlining at the MLIR level. While t</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">he
detailed design of the dialect is TBD, the next section provides links to walkthrough examples which provides a summary of the full flow as well as use of MLIR operations for OpenMP directives, and attributes for representing clauses which are constant. </span><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">The
proposed plan involves a) lowering</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;"> F18 AST with OpenMP directly to a mix of OpenMP and FIR dialects. b) converting this finally to a mix of OpenMP and LLVM dialects. This requires
that OpenMP dialect can coexist and operate with other dialects. </span><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">The design is also intended to be
modular so that other frontends (C/C++) can reuse the OpenMP dialect in the future. <br>
</span><span style="margin: 0px;"><br>
<font face="Calibri, Helvetica, sans-serif">iv) Examples</font><br>
<font face="Calibri, Helvetica, sans-serif">A few walkthroughs have been sent before [3]. These walkthroughs illustrate with an example, the flow for a few constructs (parallel, target, collapse, simd). For the parallel and target constructs will use the OpenMP
IRBuilder for lowering to LLVM IR. MLIR offers infrastructure to do loop transformations hence for the collapse clause the transformation is done inside the MLIR framework. While both LLVM and MLIR offers infrastructure for vectorisation the LLVM vectoriser
is more mature and hence LLVM is preferred. For details refer to the links [3].<br>
<br>
</font></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">v) Progress</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;"><br>
</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">i) OpenMP MLIR</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;"><br>
</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">-> Sent a few walkthroughs which illustrate the flow from AST to LLVM IR. </span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><font face="Calibri, Helvetica, sans-serif"><br>
</font></span><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">-> First patch [6] which registers the OpenMP dialect with MLIR has been submitted.</span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><font face="Calibri, Helvetica, sans-serif"><br>
</font></span><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">-> Design of a minimal dialect with a single construct (barrier) is in progress.</span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><font face="Calibri, Helvetica, sans-serif"><br>
</font></span><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">ii) OpenMP IRBuilder</span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px;"><br>
</span><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;"><font face="Calibri, Helvetica, sans-serif"><span style="margin: 0px">-><span> </span></span></font><a href="mailto:jdoerfert@anl.gov" target="_blank" rel="noopener noreferrer" style="margin: 0px; padding: 0px 1px; border-radius: 2px"><span style="margin: 0px">@Doerfert,
Johannes</span></a><font face="Calibri, Helvetica, sans-serif"><span style="margin: 0px"> has a series of patches [7] introducing preliminary support<span> </span></span></font></span><font face="Calibri, Helvetica, sans-serif" style="font-family: Calibri, Helvetica, sans-serif;"><span style="margin: 0px">for
the<span> </span></span></font><font face="Calibri, Arial, Helvetica, sans-serif" style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif;"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">OpenMP
IRBuilder which are either approved or under review. The<span> </span></span><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">initial set adds support for the parallel and barrier construct.</span><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"><br>
</span></font><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">-> Others (Roger Ferrer, Kiran) have tried it for other constructs like taskwait and</span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;"><span> </span></span><span style="font-family: Calibri, Helvetica, sans-serif; margin: 0px;">flush.</span></div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px; font-size: 12pt; background-color: rgb(255, 255, 255);">
<span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"><br>
</span></div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px; font-size: 12pt; color: rgb(32, 31, 30); background-color: rgb(255, 255, 255);">
<span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif; color: rgb(0, 0, 0)">vi) Next Steps<br>
-> Send the design to the MLIR mailing list to get approval and to enable code review.<br>
</span><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif; color: rgb(0, 0, 0)">-> Implement the accepted plan on a construct by construct basis starting with the parallel construct.<br>
</span></div>
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;">
<div style="font-size: 12pt; color: rgb(0, 0, 0);">
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px; font-size: 12pt; color: rgb(32, 31, 30); background-color: rgb(255, 255, 255);">
<span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif; color: rgb(0, 0, 0)">-> Represent construct in OpenMP MLIR</span></div>
</div>
<div style="font-size: 12pt; color: rgb(0, 0, 0);">
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px; font-size: 12pt; color: rgb(32, 31, 30); background-color: rgb(255, 255, 255);">
<span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif; color: rgb(0, 0, 0)">-> Refactor the code for the construct in OpenMP IRBuilder</span></div>
</div>
<div style="font-size: 12pt; color: rgb(0, 0, 0);">
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px; font-size: 12pt; color: rgb(32, 31, 30); background-color: rgb(255, 255, 255);">
<span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif; color: rgb(0, 0, 0)">-> Set up the translation library for OpenMP in MLIR to call the OpenMP IRBuilder</span></div>
</div>
<div style="font-size: 12pt; color: rgb(0, 0, 0);">
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px; font-size: 12pt; color: rgb(32, 31, 30); background-color: rgb(255, 255, 255);">
<span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif; color: rgb(0, 0, 0)">-> Set up the transformation from the frontend to OpenMP MLIR for this construct<br>
-> Upstream changes</span></div>
</div>
</blockquote>
<div style="font-size: 12pt; color: rgb(0, 0, 0);">
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif; margin: 0px; font-size: 12pt; color: rgb(32, 31, 30); background-color: rgb(255, 255, 255);">
<pre><font face="Calibri, Helvetica, sans-serif"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif"><pre style="background-color: rgb(255, 255, 255)"><pre style="background-color: rgb(255, 255, 255)"><font face="Calibri, Helvetica, sans-serif" style="color: rgb(0, 0, 0)"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">1) </span><span style="margin: 0px"><a href="https://drive.google.com/file/d/1vU6LsblsUYGA35B_3y9PmBvtKOTXj1Fu/view?usp=sharing" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">https://drive.google.com/file/d/1vU6LsblsUYGA35B_3y9PmBvtKOTXj1Fu/view?usp=sharing</span></a>
</span></font><span style="margin: 0px"><font color="#000000"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">2) </span></font><a href="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-August/000264.html" target="_blank" rel="noopener noreferrer" style="margin: 0px; font-family: Calibri, Arial, Helvetica, sans-serif; color: rgb(0, 0, 0)"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-August/000264.html</span></a><font color="#000000">
</font></span><span style="margin: 0px; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif;">3) Links for walkthroughs
parallel: </span><span style="margin: 0px; color: rgb(0, 0, 0)"><a href="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-August/000269.html" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="font-family: Calibri, Helvetica, sans-serif;">http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-August/000269.html</span></a></span><span style="margin: 0px; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif;">
collapse: </span><span style="margin: 0px; color: rgb(0, 0, 0)"><a href="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000273.html" target="_blank" rel="noopener noreferrer" title="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000273.html" style="margin: 0px"><span style="font-family: Calibri, Helvetica, sans-serif;">http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000273.html</span></a><br><div style="margin: 0px"><span style="font-family: Calibri, Helvetica, sans-serif;">collapse, Modification after feedback: </span><a href="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000277.html" target="_blank" rel="noopener noreferrer" title="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000277.html" style="margin: 0px"><span style="font-family: Calibri, Helvetica, sans-serif;">http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000277.html</span></a><br></div><div style="margin: 0px"><span style="font-family: Calibri, Helvetica, sans-serif;">simd: </span><a href="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000278.html" target="_blank" rel="noopener noreferrer" title="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000278.html" style="margin: 0px"><span style="font-family: Calibri, Helvetica, sans-serif;">http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000278.html</span></a><br></div></span><span style="margin: 0px; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif;">target: </span><span style="margin: 0px; color: rgb(0, 0, 0)"><a href="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000285.html" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="font-family: Calibri, Helvetica, sans-serif;">http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000285.html</span></a>
</span><span style="margin: 0px; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif;">4) </span><span style="margin: 0px; color: rgb(0, 0, 0)"><a href="https://github.com/tensorflow/mlir" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">https://github.com/tensorflow/mlir</span></a>
</span><span style="margin: 0px; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif;">5) </span><span style="margin: 0px; color: rgb(0, 0, 0)"><a href="http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-May/000197.html" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-May/000197.html</span></a>
</span><span style="margin: 0px; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif;">6) </span><a href="https://github.com/tensorflow/mlir/pull/244" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">https://github.com/tensorflow/mlir/pull/244</span></a>
<span style="margin: 0px; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif;">7) OpenMP IRBuilder (only a few included here)</span>
<a href="https://reviews.llvm.org/D69785" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">https://reviews.llvm.org/D69785</span></a>
<span style="margin: 0px"><a href="https://reviews.llvm.org/D70290" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">https://reviews.llvm.org/D70290</span></a>
<a href="https://reviews.llvm.org/D70290" target="_blank" rel="noopener noreferrer" style="margin: 0px"></a></span><a href="https://reviews.llvm.org/D69853" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">https://reviews.llvm.org/D69853</span></a>
<a href="https://reviews.llvm.org/D70290" target="_blank" rel="noopener noreferrer" style="margin: 0px"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif;">https://reviews.llvm.org/D70290</span></a><br></pre></pre></span></font></pre>
<font face="Calibri, Arial, Helvetica, sans-serif" style="color: rgb(0, 0, 0)"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">Thanks,</span></font><br>
<font face="Calibri, Arial, Helvetica, sans-serif" style="color: rgb(0, 0, 0)"><span style="margin: 0px; font-family: Calibri, Helvetica, sans-serif">Kiran</span></font></div>
<br>
</div>
</body>
</html>