[PATCH] D86003: [AMDGPU] Fix MAI ld/st hazard handling

[Stanislav Mekhanoshin via Phabricator via llvm-commits]

rampitec created this revision.
rampitec added a reviewer: arsenm.
Herald added subscribers: kerbowa, hiraditya, t-tye, tpr, dstuttard, yaxunl, nhaehnle, jvesely, kzhuravl.
Herald added a project: LLVM.
rampitec requested review of this revision.
Herald added a subscriber: wdng.

It did not process hazard for ds_permute because it does not
load or store even though it is DS.


https://reviews.llvm.org/D86003

Files:
  llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
  llvm/test/CodeGen/AMDGPU/mai-hazards.mir


Index: llvm/test/CodeGen/AMDGPU/mai-hazards.mir
===================================================================
--- llvm/test/CodeGen/AMDGPU/mai-hazards.mir
+++ llvm/test/CodeGen/AMDGPU/mai-hazards.mir
@@ -401,6 +401,45 @@
 ...
 ---
 
+# GCN-LABEL: name: accvgpr_read_write_vgpr_ds_permute
+# GCN:      V_ACCVGPR_READ_B32
+# GCN-NEXT: S_NOP 0
+# GCN-NEXT: S_NOP 0
+# GCN-NEXT: DS_PERMUTE_B32
+name:            accvgpr_read_write_vgpr_ds_permute
+body:             |
+  bb.0:
+    $vgpr0 = V_ACCVGPR_READ_B32 killed $agpr0, implicit $exec
+    $vgpr1 = DS_PERMUTE_B32 $vgpr0, $vgpr1, 0, implicit $exec
+...
+---
+
+# GCN-LABEL: name: accvgpr_read_write_vgpr_flat_load
+# GCN:      V_ACCVGPR_READ_B32
+# GCN-NEXT: S_NOP 0
+# GCN-NEXT: S_NOP 0
+# GCN-NEXT: FLAT_LOAD_DWORD
+name:            accvgpr_read_write_vgpr_flat_load
+body:             |
+  bb.0:
+    $vgpr0 = V_ACCVGPR_READ_B32 killed $agpr0, implicit $exec
+    $vgpr4 = FLAT_LOAD_DWORD $vgpr0_vgpr1, 0, 0, 0, 0, implicit $exec, implicit $flat_scr
+...
+---
+
+# GCN-LABEL: name: accvgpr_read_write_vgpr_buffer_store
+# GCN:      V_ACCVGPR_READ_B32
+# GCN-NEXT: S_NOP 0
+# GCN-NEXT: S_NOP 0
+# GCN-NEXT: BUFFER_STORE_DWORD_OFFSET
+name:            accvgpr_read_write_vgpr_buffer_store
+body:             |
+  bb.0:
+    $vgpr0 = V_ACCVGPR_READ_B32 killed $agpr0, implicit $exec
+    BUFFER_STORE_DWORD_OFFSET $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, 0, 0, 0, implicit $exec
+...
+---
+
 # GCN-LABEL: name: accvgpr_read_write_vgpr_store
 # GCN:      V_ACCVGPR_READ_B32
 # GCN-NEXT: S_NOP 0
Index: llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
===================================================================
--- llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
+++ llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
@@ -185,7 +185,9 @@
   if (SIInstrInfo::isMAI(*MI) && checkMAIHazards(MI) > 0)
     return NoopHazard;
 
-  if (MI->mayLoadOrStore() && checkMAILdStHazards(MI) > 0)
+  if ((SIInstrInfo::isVMEM(*MI) ||
+       SIInstrInfo::isFLAT(*MI) ||
+       SIInstrInfo::isDS(*MI)) && checkMAILdStHazards(MI) > 0)
     return NoopHazard;
 
   if (MI->isInlineAsm() && checkInlineAsmHazards(MI) > 0)
@@ -288,7 +290,9 @@
   if (SIInstrInfo::isMAI(*MI))
     return std::max(WaitStates, checkMAIHazards(MI));
 
-  if (MI->mayLoadOrStore())
+  if (SIInstrInfo::isVMEM(*MI) ||
+      SIInstrInfo::isFLAT(*MI) ||
+      SIInstrInfo::isDS(*MI))
     return std::max(WaitStates, checkMAILdStHazards(MI));
 
   return WaitStates;


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D86003.285775.patch
Type: text/x-patch
Size: 2488 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200814/09b4784b/attachment-0001.bin>