[flang-dev] Summary of F18 OpenMP design

Fri Dec 6 09:42:23 PST 2019

Hi Kiran,

Thank you for the detailed explanation and investigation of the OpenMP framework for lowering and code-generation.  It all looks good to me, in particular the introduction of an OpenMP MLIR dialect and the placement of OpenMP IRBuilder in the overall scheme.

Brian’s proposal for OpenMP symbol tables says that explicit symbol table entries will be created for entities that have different attributes inside of OpenMP regions.  E.g. if there’s an X in an outer scope that becomes thread private, then there’s an additional symbol table entry in the OpenMP region scope for that symbol.  That new symbol has the additional OpenMP attributes and a link back to the original entity in the outer scope.  We think this organization will simplify outlining.

I don’t recall much discussion about outlining.  Do you think OpenMP IRBuilder will drive the decisions about when and what to outline?  After outlining, will the compiler be able to run the outlined code back through the MLIR+ transformations and optimizations?  Are there situations where the outlined code presents more opportunities for optimization?

- Steve

From: Kiran Chandramohan <Kiran.Chandramohan at arm.com>
Date: Friday, November 22, 2019 at 9:26 AM
To: "flang-dev at lists.llvm.org" <flang-dev at lists.llvm.org>
Cc: "Doerfert, Johannes" <jdoerfert at anl.gov>, Roger Ferrer Ibáñez <rofirrim at gmail.com>, Stephen Scalpone <sscalpone at nvidia.com>, Richard Barton <Richard.Barton at arm.com>, Will Lovett <Will.Lovett at arm.com>
Subject: RFC: Summary of F18 OpenMP design

Hi all,

We gave a presentation [1] a few months back regarding the OpenMP design for F18 during the Flang/F18 technical call, also sent a summary mail [2] and a set of walkthroughs [3] to the mailing list. We received some feedback and have incorporated that into the design. A summary of the design and status of the OpenMP implementation in F18 from lowering of the parse tree to LLVM IR generation is presented in this mail. (Note that Semantic and Structural checks are not covered. Refer to Gary's biweekly mail for the status.)

The proposed design can be seen in slide 10 of the presentation [1]. The design summary is as follows.
i) Uses the following two components.
a) MLIR [4]: Necessary since MLIR has already been chosen as the framework for building the Fortran IR (FIR) for Flang. By using MLIR for OpenMP we have a common representation for OpenMP and Fortran constructs in the MLIR framework and thereby take advantage of optimisations and avoid black boxes.
b) OpenMP IRBuilder [5]: For sharing of OpenMP codegen with Clang.

ii) Current and Proposed Flow
a) The current sequential code flow in Flang (Slide 5) of the presentation [1] can be summarised as follows,
[Fortran code] -> Parser -> [AST] -> Lowering -> [FIR MLIR] -> Conversion -> [LLVM MLIR] -> Translation -> [LLVM IR]
b) The modified flow with OpenMP (Slide 10) will have lowering of the AST to a mix of FIR and OpenMP dialects. These are then optimised and finally converted to mix of OpenMP and LLVM MLIR dialects. The mix is translated to LLVM IR using the existing translation library for LLVM MLIR and the OpenMP IRBuilder currently under construction.
[Fortran code] -> Parser -> [AST] -> Lowering -> [FIR + OpenMP MLIR] -> Conversion -> [LLVM + OpenMP MLIR] -> Translation (Use OpenMP IRBuilder) -> [LLVM IR]
c) The MLIR infrastructure provides a lot of optimisation passes for loops. It is desirable that we take advantage of some of these. But the LLVM infrastructure also provides several optimisations. So there exist some questions regarding where the optimisations should be carried out. We will decide on which framework to choose only after some experimentation. If we decide that the OpenMP construct (for e.g. collapse) can be handled fully in MLIR and that is the best place to do it (based on experiments) then we will not use the OpenMP IRBuilder for these constructs.

iii) OpenMP MLIR dialect
The proposed plan involves writing an MLIR dialect for OpenMP. Operations of the dialect will be a mix of fine and coarse-grained. e.g. Coarse: omp.parallel, omp.target, Fine: omp.flush. The operations in MLIR can have regions, hence there is no need for outlining at the MLIR level. While the detailed design of the dialect is TBD, the next section provides links to walkthrough examples which provides a summary of the full flow as well as use of MLIR operations for OpenMP directives, and attributes for representing clauses which are constant.  The proposed plan involves a) lowering F18 AST with OpenMP directly to a mix of OpenMP and FIR dialects. b) converting this finally to a mix of OpenMP and LLVM dialects. This requires that OpenMP dialect can coexist and operate with other dialects. The design is also intended to be modular so that other frontends (C/C++) can reuse the OpenMP dialect in the future.

iv) Examples
A few walkthroughs have been sent before [3]. These walkthroughs illustrate with an example, the flow for a few constructs (parallel, target, collapse, simd). For the parallel and target constructs will use the OpenMP IRBuilder for lowering to LLVM IR. MLIR offers infrastructure to do loop transformations hence for the collapse clause the transformation is done inside the MLIR framework. While both LLVM and MLIR offers infrastructure for vectorisation the LLVM vectoriser is more mature and hence LLVM is preferred. For details refer to the links [3].

v) Progress
i) OpenMP MLIR
-> Sent a few walkthroughs which illustrate the flow from AST to LLVM IR.
-> First patch [6] which registers the OpenMP dialect with MLIR has been submitted.
-> Design of a minimal dialect with a single construct (barrier) is in progress.
ii) OpenMP IRBuilder
-> @Doerfert, Johannes<mailto:jdoerfert at anl.gov> has a series of patches [7] introducing preliminary support for the OpenMP IRBuilder which are either approved or under review. The initial set adds support for the parallel and barrier construct.
-> Others (Roger Ferrer, Kiran) have tried it for other constructs like taskwait and flush.

vi) Next Steps
-> Send the design to the MLIR mailing list to get approval and to enable code review.
-> Implement the accepted plan on a construct by construct basis starting with the parallel construct.
-> Represent construct in OpenMP MLIR
-> Refactor the code for the construct in OpenMP IRBuilder
-> Set up the translation library for OpenMP in MLIR to call the OpenMP IRBuilder
-> Set up the transformation from the frontend to OpenMP MLIR for this construct
-> Upstream changes

1) https://drive.google.com/file/d/1vU6LsblsUYGA35B_3y9PmBvtKOTXj1Fu/view?usp=sharing

2) http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-August/000264.html

3) Links for walkthroughs

parallel: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-August/000269.html

collapse: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000273.html

collapse, Modification after feedback: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000277.html

simd: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000278.html

target: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000285.html

4) https://github.com/tensorflow/mlir

5) http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-May/000197.html

6) https://github.com/tensorflow/mlir/pull/244

7) OpenMP IRBuilder (only a few included here)

https://reviews.llvm.org/D69785

https://reviews.llvm.org/D70290

https://reviews.llvm.org/D69853

https://reviews.llvm.org/D70290
Thanks,
Kiran

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/flang-dev/attachments/20191206/fd3b3e0d/attachment-0001.html>