[flang-dev] Summary of F18 OpenMP design

Kiran Chandramohan via flang-dev flang-dev at lists.llvm.org
Sun Dec 8 06:51:04 PST 2019 

Hello Steve,

Thanks for your mail.

Yes, we will use the OpenMPIR builder to outline. I have briefly mentioned this in the parallel construct example (including that below for your reference). Note that using the OpenMP IRBuilder will be the last step before LLVM IR generation. After this point, there is no going back to MLIR. Any transformation or optimisation in MLIR has to be done before this step. This can involve constant propagation and other code motion to reduce the bad effects that outlining brings. But this is new code that will have to be written.

Example 1: Parallel construct

1) Example OpenMP code

<Fortran code>

!$omp parallel

c = a + b

!$omp end parallel

<More Fortran code>




2) Parse tree (omitted)


3) The first lowering will be to a mix of FIR dialect and OpenMP dialect. The OpenMP dialect has an operation called parallel with a nested region of code. The nested region will have FIR (and standard dialect) operations.


Mlir.region(…) {

%1 = fir.x(…)

…

%20 = omp.parallel {

        %1 = addf %2, %3 : f32

      }

%21 = <more fir>

…

}



4) The next lowering will be to OpenMP and LLVM dialect

Mlir.region(…) {

%1 = llvm.xyz<http://llvm.xyz/>(...)

…

%20 = omp.parallel {

        %1 = llvm.fadd %2, %3 : !llvm.float

      }

 %21 = <more llvm dialect>

…

}



5) The next conversion will be to LLVM IR. Here the OpenMP dialect will be lowered using the OpenMP IRBuilder and the translation library of the LLVM dialect. The IR Builder will see that there is a region under the OpenMP construct omp.parallel. It will collect all the basic blocks inside that region and then generate outlined code using those basic blocks. Suitable calls will be inserted to the OpenMP API.



define @outlined_parallel_fn(...)

{

  ....

  %1 = fadd float %2, %3

  ...

}



define @xyz(…)

{

  %1 = alloca float

  ....

  call kmpc_fork_call(...,outlined_parallel_fn,...)

}


Regards,
Kiran
________________________________
From: Stephen Scalpone <sscalpone at nvidia.com>
Sent: 06 December 2019 17:42
To: Kiran Chandramohan <Kiran.Chandramohan at arm.com>; flang-dev at lists.llvm.org <flang-dev at lists.llvm.org>
Cc: Doerfert, Johannes <jdoerfert at anl.gov>; Roger Ferrer Ibáñez <rofirrim at gmail.com>; Richard Barton <Richard.Barton at arm.com>; Will Lovett <Will.Lovett at arm.com>
Subject: Re: Summary of F18 OpenMP design


Hi Kiran,



Thank you for the detailed explanation and investigation of the OpenMP framework for lowering and code-generation.  It all looks good to me, in particular the introduction of an OpenMP MLIR dialect and the placement of OpenMP IRBuilder in the overall scheme.



Brian’s proposal for OpenMP symbol tables says that explicit symbol table entries will be created for entities that have different attributes inside of OpenMP regions.  E.g. if there’s an X in an outer scope that becomes thread private, then there’s an additional symbol table entry in the OpenMP region scope for that symbol.  That new symbol has the additional OpenMP attributes and a link back to the original entity in the outer scope.  We think this organization will simplify outlining.



I don’t recall much discussion about outlining.  Do you think OpenMP IRBuilder will drive the decisions about when and what to outline?  After outlining, will the compiler be able to run the outlined code back through the MLIR+ transformations and optimizations?  Are there situations where the outlined code presents more opportunities for optimization?



- Steve





From: Kiran Chandramohan <Kiran.Chandramohan at arm.com>
Date: Friday, November 22, 2019 at 9:26 AM
To: "flang-dev at lists.llvm.org" <flang-dev at lists.llvm.org>
Cc: "Doerfert, Johannes" <jdoerfert at anl.gov>, Roger Ferrer Ibáñez <rofirrim at gmail.com>, Stephen Scalpone <sscalpone at nvidia.com>, Richard Barton <Richard.Barton at arm.com>, Will Lovett <Will.Lovett at arm.com>
Subject: RFC: Summary of F18 OpenMP design



Hi all,

We gave a presentation [1] a few months back regarding the OpenMP design for F18 during the Flang/F18 technical call, also sent a summary mail [2] and a set of walkthroughs [3] to the mailing list. We received some feedback and have incorporated that into the design. A summary of the design and status of the OpenMP implementation in F18 from lowering of the parse tree to LLVM IR generation is presented in this mail. (Note that Semantic and Structural checks are not covered. Refer to Gary's biweekly mail for the status.)

The proposed design can be seen in slide 10 of the presentation [1]. The design summary is as follows.
i) Uses the following two components.
a) MLIR [4]: Necessary since MLIR has already been chosen as the framework for building the Fortran IR (FIR) for Flang. By using MLIR for OpenMP we have a common representation for OpenMP and Fortran constructs in the MLIR framework and thereby take advantage of optimisations and avoid black boxes.
b) OpenMP IRBuilder [5]: For sharing of OpenMP codegen with Clang.

ii) Current and Proposed Flow
a) The current sequential code flow in Flang (Slide 5) of the presentation [1] can be summarised as follows,
[Fortran code] -> Parser -> [AST] -> Lowering -> [FIR MLIR] -> Conversion -> [LLVM MLIR] -> Translation -> [LLVM IR]
b) The modified flow with OpenMP (Slide 10) will have lowering of the AST to a mix of FIR and OpenMP dialects. These are then optimised and finally converted to mix of OpenMP and LLVM MLIR dialects. The mix is translated to LLVM IR using the existing translation library for LLVM MLIR and the OpenMP IRBuilder currently under construction.
[Fortran code] -> Parser -> [AST] -> Lowering -> [FIR + OpenMP MLIR] -> Conversion -> [LLVM + OpenMP MLIR] -> Translation (Use OpenMP IRBuilder) -> [LLVM IR]
c) The MLIR infrastructure provides a lot of optimisation passes for loops. It is desirable that we take advantage of some of these. But the LLVM infrastructure also provides several optimisations. So there exist some questions regarding where the optimisations should be carried out. We will decide on which framework to choose only after some experimentation. If we decide that the OpenMP construct (for e.g. collapse) can be handled fully in MLIR and that is the best place to do it (based on experiments) then we will not use the OpenMP IRBuilder for these constructs.

iii) OpenMP MLIR dialect
The proposed plan involves writing an MLIR dialect for OpenMP. Operations of the dialect will be a mix of fine and coarse-grained. e.g. Coarse: omp.parallel, omp.target, Fine: omp.flush. The operations in MLIR can have regions, hence there is no need for outlining at the MLIR level. While the detailed design of the dialect is TBD, the next section provides links to walkthrough examples which provides a summary of the full flow as well as use of MLIR operations for OpenMP directives, and attributes for representing clauses which are constant.  The proposed plan involves a) lowering F18 AST with OpenMP directly to a mix of OpenMP and FIR dialects. b) converting this finally to a mix of OpenMP and LLVM dialects. This requires that OpenMP dialect can coexist and operate with other dialects. The design is also intended to be modular so that other frontends (C/C++) can reuse the OpenMP dialect in the future.

iv) Examples
A few walkthroughs have been sent before [3]. These walkthroughs illustrate with an example, the flow for a few constructs (parallel, target, collapse, simd). For the parallel and target constructs will use the OpenMP IRBuilder for lowering to LLVM IR. MLIR offers infrastructure to do loop transformations hence for the collapse clause the transformation is done inside the MLIR framework. While both LLVM and MLIR offers infrastructure for vectorisation the LLVM vectoriser is more mature and hence LLVM is preferred. For details refer to the links [3].

v) Progress
i) OpenMP MLIR
-> Sent a few walkthroughs which illustrate the flow from AST to LLVM IR.
-> First patch [6] which registers the OpenMP dialect with MLIR has been submitted.
-> Design of a minimal dialect with a single construct (barrier) is in progress.
ii) OpenMP IRBuilder
-> @Doerfert, Johannes<mailto:jdoerfert at anl.gov> has a series of patches [7] introducing preliminary support for the OpenMP IRBuilder which are either approved or under review. The initial set adds support for the parallel and barrier construct.
-> Others (Roger Ferrer, Kiran) have tried it for other constructs like taskwait and flush.



vi) Next Steps
-> Send the design to the MLIR mailing list to get approval and to enable code review.
-> Implement the accepted plan on a construct by construct basis starting with the parallel construct.

-> Represent construct in OpenMP MLIR

-> Refactor the code for the construct in OpenMP IRBuilder

-> Set up the translation library for OpenMP in MLIR to call the OpenMP IRBuilder

-> Set up the transformation from the frontend to OpenMP MLIR for this construct
-> Upstream changes

1) https://drive.google.com/file/d/1vU6LsblsUYGA35B_3y9PmBvtKOTXj1Fu/view?usp=sharing

2) http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-August/000264.html

3) Links for walkthroughs

parallel: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-August/000269.html

collapse: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000273.html

collapse, Modification after feedback: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000277.html

simd: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000278.html

target: http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-September/000285.html

4) https://github.com/tensorflow/mlir

5) http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-May/000197.html

6) https://github.com/tensorflow/mlir/pull/244

7) OpenMP IRBuilder (only a few included here)

https://reviews.llvm.org/D69785

https://reviews.llvm.org/D70290

https://reviews.llvm.org/D69853

https://reviews.llvm.org/D70290

Thanks,
Kiran



________________________________
This email message is for the sole use of the intended recipient(s) and may contain confidential information.  Any unauthorized review, use, disclosure or distribution is prohibited.  If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/flang-dev/attachments/20191208/0dbae9af/attachment-0001.html>

More information about the flang-dev mailing list