[Mlir-commits] [llvm] [mlir] [MLIR][OpenMP] Lowering nontemporal clause to LLVM IR for SIMD directive (PR #118751)

[Kiran Chandramohan]

kiranchandramohan wrote:

> Thanks for taking a look Kiran.
> 
> I don't think we can do nontemporal(%18) because %18 is not defined at the point it is being refereed to. But we could have a flang pass which adds a non-temporal attribute to fir.load and fir.store, and that attribute could be propagated to the llvm operation.
> 
> For example,
> 
> ```
> 
> 
> For the following example, it will be moving from nontemporal(%3) to nontemporal(%18)
> 
>     omp.simd nontemporal(%3 : !fir.ref<!fir.box<!fir.heap<!fir.array<?xi32>>>>) private(@_QFsbEi_private_i32 %2 -> %arg2 : !fir.ref<i32>) {
>       omp.loop_nest (%arg3) : i32 = (%c1_i32) to (%c100_i32) inclusive step (%c1_i32) {
>         %11 = fir.declare %arg2 {uniq_name = "_QFsbEi"} : (!fir.ref<i32>) -> !fir.ref<i32>
>         fir.store %arg3 to %11 : !fir.ref<i32>
>         %12 = fir.load %3 : !fir.ref<!fir.box<!fir.heap<!fir.array<?xi32>>>>
>         %13 = fir.load %11 : !fir.ref<i32>
>         %14 = fir.convert %13 : (i32) -> i64
>         %15 = fir.box_addr %12 : (!fir.box<!fir.heap<!fir.array<?xi32>>>) -> !fir.heap<!fir.array<?xi32>>
>         %16:3 = fir.box_dims %12, %c0 : (!fir.box<!fir.heap<!fir.array<?xi32>>>, index) -> (index, index, index)
>         %17 = fir.shape_shift %16#0, %16#1 : (index, index) -> !fir.shapeshift<1>
>         %18 = fir.array_coor %15(%17) %14 : (!fir.heap<!fir.array<?xi32>>, !fir.shapeshift<1>, i64) -> !fir.ref<i32>
>         %19 = fir.load %18 : !fir.ref<i32> nontemporal
>         %20 = fir.load %4 : !fir.ref<i32>
>         %21 = arith.addi %19, %20 : i32
>         fir.store %21 to %18 : !fir.ref<i32> nontemporal
>         omp.yield
>       }
> ```

Yes, transferring to the correct loads and stores in a Flang pass and then handling the load/store with the attributes in translation/OpenMP seems to be a reasonable approach. 

There are a few ways of doing this:
1. We could consider adding an OpenMP dialect attribute for nontemporal loads and stores and then translate it to the nontemporal metadata during OpenMP translation
     -> at convertOperation of simd
     -> using the amendOperation interface.
2. We can add a nontemporal attribute to FIR loads/store and since LLVM dialect loads and stores have `UnitAttr:$nontemporal`, we could set this during the FIR to LLVM conversion.
3. Add an LLVM dialect attribute for nontemporal and then use that in fir and llvm loads and store.

The issue with attributes is that there is a danger that they can be dropped. But I assume we can guarantee retaining them in Flang.  If losing attributes is a concern, we could also consider creating a wrapper operation for the fir.load or fir.store. 
```
%t = omp.nontemporal {
  %1 = fir.load
  omp.yield %1
}
```

https://github.com/llvm/llvm-project/pull/118751