[all-commits] [llvm/llvm-project] 9cbeb9: [flang][hlfir] move intrinsic lowering out of Buff...

[Tom Eccles via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 9cbeb97024515e6c1fca506623c384b4876ac231
      https://github.com/llvm/llvm-project/commit/9cbeb97024515e6c1fca506623c384b4876ac231
  Author: Tom Eccles <tom.eccles at arm.com>
  Date:   2023-03-17 (Fri, 17 Mar 2023)

  Changed paths:
    M flang/include/flang/Optimizer/HLFIR/Passes.h
    M flang/include/flang/Optimizer/HLFIR/Passes.td
    M flang/lib/Optimizer/HLFIR/Transforms/BufferizeHLFIR.cpp
    M flang/lib/Optimizer/HLFIR/Transforms/CMakeLists.txt
    A flang/lib/Optimizer/HLFIR/Transforms/LowerHLFIRIntrinsics.cpp
    R flang/test/HLFIR/matmul-bufferization.fir
    A flang/test/HLFIR/matmul-lowering.fir
    R flang/test/HLFIR/sum-bufferization.fir
    A flang/test/HLFIR/sum-lowering.fir
    R flang/test/HLFIR/transpose-bufferization.fir
    A flang/test/HLFIR/transpose-lowering.fir

  Log Message:
  -----------
  [flang][hlfir] move intrinsic lowering out of BufferizeHLFIR

This move is useful for a few reasons:
  - It is easier to see what the intrinsic lowering is doing when the
    operations it creates are not immediately lowered
  - When lowering a HLFIR intrinsic generates an operation which needs
    to be lowered by another pattern matcher in the same pass, MLIR will
    run that other substitution before validating and finalizing the
    original changes. This means that the erasure of operations is not
    yet visible to subsequent matchers, which hugely complicates
    transformations (in this case, hlfir.exprs cannot be rewritten
    because they are still used by the now-erased HLFIR intrinsic op.

Reviewed By: jeanPerier

Differential Revision: https://reviews.llvm.org/D145502


  Commit: 3e0f6b2eb4699e37cc7009d0a2056d85ca15d64c
      https://github.com/llvm/llvm-project/commit/3e0f6b2eb4699e37cc7009d0a2056d85ca15d64c
  Author: Tom Eccles <tom.eccles at arm.com>
  Date:   2023-03-17 (Fri, 17 Mar 2023)

  Changed paths:
    M flang/lib/Optimizer/HLFIR/Transforms/LowerHLFIRIntrinsics.cpp

  Log Message:
  -----------
  [flang][hlfir] clean up intrinsic arg extended values

Depends on D145502

Reviewed By: jeanPerier

Differential Revision: https://reviews.llvm.org/D145503


  Commit: e29d1d0f80b0ee39115f68da61ca1417f315a322
      https://github.com/llvm/llvm-project/commit/e29d1d0f80b0ee39115f68da61ca1417f315a322
  Author: Tom Eccles <tom.eccles at arm.com>
  Date:   2023-03-17 (Fri, 17 Mar 2023)

  Changed paths:
    A flang/test/HLFIR/mul_transpose.f90

  Log Message:
  -----------
  [flang][hlfir] add a test chaining intrinsics

This test checks all of the parts of intrinsic lowering work together,
and makes sure that we can pass the hlfir.expr result of one intrinsic
as an argument to another.

Depends on D145503

Reviewed By: jeanPerier

Differential Revision: https://reviews.llvm.org/D145504


  Commit: 49bd444fc3617a140ef67047d756c4d652a2a835
      https://github.com/llvm/llvm-project/commit/49bd444fc3617a140ef67047d756c4d652a2a835
  Author: Tom Eccles <tom.eccles at arm.com>
  Date:   2023-03-17 (Fri, 17 Mar 2023)

  Changed paths:
    M flang/docs/HighLevelFIR.md
    M flang/include/flang/Optimizer/HLFIR/HLFIROps.td
    M flang/lib/Optimizer/HLFIR/IR/HLFIROps.cpp
    M flang/test/HLFIR/invalid.fir
    A flang/test/HLFIR/matmul_transpose.fir

  Log Message:
  -----------
  [flang][hlfir] add hlfir.matmul_transpose operation

This operation will be used to transform MATMUL(TRANSPOSE(a), b). The
transformation will go in the following stages:
        1. Lowering to hlfir.transpose and hlfir.matmul
        2. Canonicalise to hlfir.matmul_transpose
        3. hlfir.matmul_transpose will be lowered to FIR as a new runtime
          library call

Step 2 (and this operation) are included for consistency with the other
hlfir intrinsic operations and to avoid mixing concerns in the intrinsic
lowering pass.

In step 3, a new runtime library call is used because this operation is
most easily implemented in one go (the transposed indexing actually
makes the indexing simpler than for a normal matrix multiplication). In
the long run, it is intended that HLFIR will allow the same buffer
to be shared between different runtime calls without temporary
allocations, but in this specific case we can do even better than that
with a dedicated implementation.

This should speed up galgel from SPEC2000 (but this hadn't been tested
yet). The optimization was implemented in Classic Flang.

Reviewed By: vzakhari

Differential Revision: https://reviews.llvm.org/D145957


  Commit: a351a60ebae456735ec32808f311a6e9cf5e751e
      https://github.com/llvm/llvm-project/commit/a351a60ebae456735ec32808f311a6e9cf5e751e
  Author: Tom Eccles <tom.eccles at arm.com>
  Date:   2023-03-17 (Fri, 17 Mar 2023)

  Changed paths:
    M flang/include/flang/Optimizer/HLFIR/HLFIROps.h
    M flang/include/flang/Optimizer/HLFIR/HLFIROps.td
    M flang/lib/Optimizer/HLFIR/IR/HLFIROps.cpp
    M flang/test/HLFIR/mul_transpose.f90

  Log Message:
  -----------
  [flang][hlfir] add matmul canonicalizer

hlfir.matmul_transpose will be lowered to a new runtime call.

A canonicalizer was chosen because
  - Alternative: a new pass for rewriting chained intrinsics - this
    would add a lot of unnecessary boilerplate.
  - Alternative: including this in the HLFIR Intrinsic Lowering pass -
    I wanted to separate these two concerns: not adding a second purpose
    complicating the intrinsic lowering pass.

With this change, the MLIR built-in canonicalization pass should be run
before the HLFIR Intrinsic Lowering pass.

Depends on D145504, D145957

Reviewed By: clementval, vzakhari

Differential Revision: https://reviews.llvm.org/D145959


  Commit: 4ff8ba72b58328ebf6e8eb8c10a428eece73c89f
      https://github.com/llvm/llvm-project/commit/4ff8ba72b58328ebf6e8eb8c10a428eece73c89f
  Author: Tom Eccles <tom.eccles at arm.com>
  Date:   2023-03-17 (Fri, 17 Mar 2023)

  Changed paths:
    A flang/include/flang/Runtime/matmul-transpose.h
    M flang/runtime/CMakeLists.txt
    A flang/runtime/matmul-transpose.cpp
    M flang/unittests/Runtime/CMakeLists.txt
    A flang/unittests/Runtime/MatmulTranspose.cpp

  Log Message:
  -----------
  [flang] add fused matmul-transpose to the runtime

This fused operation should run a lot faster than first transposing the
lhs array and then multiplying the matrices separately.

Based on flang/runtime/matmul.cpp

Depends on D145959

Reviewed By: klausler

Differential Revision: https://reviews.llvm.org/D145960


  Commit: 43eb96cab8f734a740b86c553f48f96fa9d19edc
      https://github.com/llvm/llvm-project/commit/43eb96cab8f734a740b86c553f48f96fa9d19edc
  Author: Tom Eccles <tom.eccles at arm.com>
  Date:   2023-03-17 (Fri, 17 Mar 2023)

  Changed paths:
    M flang/include/flang/Optimizer/Builder/Runtime/Transformational.h
    M flang/lib/Optimizer/Builder/IntrinsicCall.cpp
    M flang/lib/Optimizer/Builder/Runtime/Transformational.cpp
    M flang/lib/Optimizer/HLFIR/Transforms/LowerHLFIRIntrinsics.cpp
    M flang/test/HLFIR/mul_transpose.f90

  Log Message:
  -----------
  [flang][hlfir] lower hlfir.matmul_transpose to runtime call

Depends on D145960

Reviewed By: jeanPerier

Differential Revision: https://reviews.llvm.org/D145961


Compare: https://github.com/llvm/llvm-project/compare/acdb199a2f50...43eb96cab8f7