[flang-commits] [flang] [flang][Multi-Image] Moving Mutli-image lowering to PRIF into the MIF dialect (PR #161179)

[Dan Bonachea via flang-commits]

================
@@ -0,0 +1,263 @@
+//===-- MIFOps.td - MIF operation definitions --------------*- tablegen -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// Definition of the MIF dialect operations
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef FORTRAN_DIALECT_MIF_MIF_OPS
+#define FORTRAN_DIALECT_MIF_MIF_OPS
+
+include "flang/Optimizer/Dialect/MIF/MIFDialect.td"
+include "flang/Optimizer/Dialect/FIRTypes.td"
+include "flang/Optimizer/Dialect/FIRAttr.td"
+
+class mif_Op<string mnemonic, list<Trait> traits>
+    : Op<MIFDialect, mnemonic, traits>;
+
+//===----------------------------------------------------------------------===//
+// Initialization and Finalization
+//===----------------------------------------------------------------------===//
+
+def mif_InitOp : mif_Op<"init", []> {
+  let summary = "Initialize the parallel environment";
+  let description = [{This operation will initialize the parallel environment}];
+
+  let results = (outs I32:$stat);
+  let assemblyFormat = "`->` type($stat) attr-dict";
+}
+
+//===----------------------------------------------------------------------===//
+// Image Queries
+//===----------------------------------------------------------------------===//
+
+def mif_NumImagesOp : mif_Op<"num_images", [AttrSizedOperandSegments]> {
+  let summary = "Query the number of images in the specified or current team";
+  let description = [{
+    This operation query the number of images in the specified or current
+    team and can be called with 3 differents way :
+    - `num_images()`
+    - `num_images(team)`
+    - `num_images(team_number)`
+
+    Arguments:
+    - `team` : Shall be a scalar of type `team_type` from the `ISO_FORTRAN_ENV`
+            module with a value that identifies the current or ancestor team.
+    - `team_number` :  Shall be an integer scalar. It shall identify the
+            initial team or a sibling team of the current team.
+
+    Result Value: The number of images in the specified team, or in the current
+    team if no team is specified.
+  }];
+
+  let arguments = (ins Optional<AnyInteger>:$team_number,
+                       Optional<AnyRefOrBoxType>:$team);
+  let results = (outs I32:$res);
+
+  let builders = [OpBuilder<(ins CArg<"mlir::Value", "{}">:$teamArg)>];
+
+  let hasVerifier = 1;
+  let assemblyFormat = [{
+    ( `team_number` $team_number^ )? 
+    ( `team` $team^ )? 
+    attr-dict `:` functional-type(operands, results)
+  }];
+}
+
+def mif_ThisImageOp : mif_Op<"this_image", [AttrSizedOperandSegments]> {
+  let summary = "Determine the image index of the current image";
+  let description = [{
+    Arguments:
+    - `coarray` :  Shall be a coarray of any type.
+    - `dim` : Shall be an integer scalar. Its value shall be in the range of
+          1 <= DIM <= N, where N is the corank of the coarray.
+    - `team`(optional) : Shall be a scalar of type `team_type` from
+          ISO_FORTRAN_ENV. If the `coarray` is present, it shall be
+          established in that team.
+
+    Results:
+    - Case(1) : The result of `this_image([team])` is a scalar with a value
+          equal to the index of the image in the current or specified team.
+    - Case(2) : The result of `this_image(coarray [,team])` is the sequence of
+          cosubscript values for `coarray`.
+    - Case(3) : The result of `this_image(coarray, dim [,team])` is the value of
+          cosubscript `dim` in the sequence of cosubscript values for `coarray`.
+
+    Example:
+    ```fortran
+      REAL :: A[10, 0:9, 0:*]
+    ```
+    If we take a look on the example and we are on image 5, `this_image` has the
+    value 5, `this_image(A)` has the value [5, 0, 0].
+  }];
+
+  let arguments = (ins Optional<fir_BoxType>:$coarray,
+      Optional<AnyInteger>:$dim, Optional<AnyRefOrBoxType>:$team);
+  let results = (outs I32:$res);
+
+  let builders = [OpBuilder<(ins "mlir::Value":$coarray, "mlir::Value":$team)>,
+                  OpBuilder<(ins "mlir::Value":$team)>];
+
+  let hasVerifier = 1;
+  let assemblyFormat = [{
+    ( `coarray` $coarray^ )? 
+    ( `team` $team^ )? 
+    ( `dim` $dim^ )? 
+    attr-dict `:` functional-type(operands, results)
+  }];
+}
+
+//===----------------------------------------------------------------------===//
+// Synchronization
+//===----------------------------------------------------------------------===//
+
+def mif_SyncAllOp : mif_Op<"sync_all", [AttrSizedOperandSegments,
+                                        MemoryEffects<[MemWrite]>]> {
----------------
bonachea wrote:

All three SYNC operations in this section (`sync_all`, `sync_images` and `sync_memory`) are Fortran "image control statements" (see F23 11.7.1). This means they end a Fortran "segment" and start another (see F23 11.7.2), which affects the memory consistency semantics of the multi-image execution. 

One practical impact of this is to restrict the types of optimizations that can safely be performed on surrounding operations that might access memory reachable from a coarray (which generally includes any variable directly in a coarray _or_ any variable with a TARGET attribute). So for example in the absence of very sophisticated analysis, the LLVM optimizer should not be permitted to effectively hoist any access operations touching such locations across an image control statement like `sync all`.

The parallel runtime library will only ensure the image synchronization semantics of the operation and inhibit hardware-level reordering. It's the compiler's responsibility to ensure that optimizations don't move surrounding memory access operations across the image control statement in ways that could break the Fortran memory model. Such violations _probably_ wouldn't affect visible behavior today, but could become a visible problem once we add coindexed access.

I found only [limited documentation](https://mlir.llvm.org/docs/Rationale/SideEffectsAndSpeculation/) on `mlir::MemoryEffects`, so I'm not sure if the dialect is a sufficient and proper place to represent this category of constraint, or if it belongs elsewhere. However one conservative way to enforce the required constraint _might_ be to set the `MemoryEffects` of dialect operations corresponding to image control statements such that compiler analysis assumes the operation effectively has a side-effect of reading and writing **_all_** memory locations (not just those directly referenced by the arguments).

@jeanPerier Can this be expressed by setting attributes like `MemoryEffects<[MemRead,MemWrite]>` on dialect operations corresponding to image control statements? Or do we need something even more aggressive like `MemoryEffects<[MemRead<DefaultResource, 0, FullEffect>,MemWrite<DefaultResource, 0, FullEffect>]>`, or something else entirely?






https://github.com/llvm/llvm-project/pull/161179