[Mlir-commits] [clang] [llvm] [mlir] [AMDGPU] Implement AsyncMark Stages (PR #220442)

[Ryan Mitchell]

RyanRio wrote:

Do you want something like this --

---
name:            loop_carried_mark
tracksRegLiveness: true
body:             |
  ; CHECK-LABEL: name: loop_carried_mark
  ; CHECK: bb.0:
  ; CHECK-NEXT:   successors: %bb.1(0x80000000)
  ; CHECK-NEXT:   liveins: $vgpr0_vgpr1, $vgpr2, $sgpr0
  ; CHECK-NEXT: {{  $}}
  ; CHECK-NEXT:   S_WAIT_LOADCNT_DSCNT .Loadcnt_0_Dscnt_0
  ; CHECK-NEXT:   S_WAIT_KMCNT 0
  ; CHECK-NEXT:   S_BRANCH %bb.1
  ; CHECK-NEXT: {{  $}}
  ; CHECK-NEXT: bb.1:
  ; CHECK-NEXT:   successors: %bb.1(0x40000000), %bb.2(0x40000000)
  ; CHECK-NEXT:   liveins: $vgpr0_vgpr1, $vgpr2, $sgpr0
  ; CHECK-NEXT: {{  $}}
  ; CHECK-NEXT:   WAIT_ASYNCMARK 0, .GLOBAL_LOAD_ASYNC_TO_LDS
  ; CHECK-NEXT:   S_WAIT_DSCNT 0
  ; CHECK-NEXT:   S_WAIT_ASYNCCNT 0, implicit-def $asynccnt, implicit $asynccnt
  ; CHECK-NEXT:   $vgpr3 = DS_READ_B32_gfx9 $vgpr2, 0, 0, implicit $exec :: (load (s32), addrspace 3)
  ; CHECK-NEXT:   GLOBAL_LOAD_ASYNC_TO_LDS_B32 $vgpr2, $vgpr0_vgpr1, 0, 0, implicit-def $asynccnt, implicit $exec, implicit $asynccnt :: (load (s32), addrspace 1), (store (s32), addrspace 3)
  ; CHECK-NEXT:   ASYNCMARK .GLOBAL_LOAD_ASYNC_TO_LDS
  ; CHECK-NEXT:   S_CMP_EQ_U32 $sgpr0, 0, implicit-def $scc
  ; CHECK-NEXT:   S_CBRANCH_SCC1 %bb.1, implicit $scc
  ; CHECK-NEXT:   S_BRANCH %bb.2
  ; CHECK-NEXT: {{  $}}
  ; CHECK-NEXT: bb.2:
  ; CHECK-NEXT:   S_ENDPGM 0
  bb.0:
    successors: %bb.1
    liveins: $vgpr0_vgpr1, $vgpr2, $sgpr0

    S_BRANCH %bb.1

  bb.1:
    successors: %bb.1, %bb.2
    liveins: $vgpr0_vgpr1, $vgpr2, $sgpr0

    WAIT_ASYNCMARK 0, 2
    $vgpr3 = DS_READ_B32_gfx9 $vgpr2, 0, 0, implicit $exec :: (load (s32), addrspace 3)
    GLOBAL_LOAD_ASYNC_TO_LDS_B32 $vgpr2, $vgpr0_vgpr1, 0, 0, implicit-def $asynccnt, implicit $exec, implicit $asynccnt :: (load (s32), addrspace 1), (store (s32), addrspace 3)
    ASYNCMARK 2
    S_CMP_EQ_U32 $sgpr0, 0, implicit-def $scc
    S_CBRANCH_SCC1 %bb.1, implicit $scc
    S_BRANCH %bb.2

  bb.2:
    S_ENDPGM 0
...

https://github.com/llvm/llvm-project/pull/220442