[PATCH] D59036: Memory writes overlap in the pipelined loop

[Yan Luo via Phabricator via llvm-commits]

yan_luo created this revision.
yan_luo added a reviewer: bcahoon.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

The original loop is like:

  %15 = …

LOOP:

  %26 = PHI %15, %31 
  …
  STORE -> [%26 + 32, %26 + 32 + 64]  <- 64 is the access size
  …
  STORE -> [%26 + 0, %26 + 0 + 64]
  …
  STORE -> [%26 + 62, %26 + 62 + 64]
  …
  STORE -> [%26 + 94, %26 + 94 + 64]
  …
  %31 = ADD %26, 124

LOOP_END

The software pipelined loop is like:

  %15 = …
  %234 = COPY %15

PROLOG:

  …
  STORE -> [%234 + 32, %234 + 32 + 64]
  …
  STORE -> [%234 + 0, %234 + 0 + 64]
  …
  STORE -> [%234 + 62, %234 + 62 + 64]
  …
  %227 = ADD %234, 124

KERNEL:

  %291 = PHI %227, %241
  %292 = PHI %15, %291
  …
  %303 = COPY %291
  %241 = ADD %303, 124
  …
  STORE -> [%303 + 32, %303 + 32 + 64]
  …
  STORE -> [%303 + 0, %303 + 0 + 64]
  …
  STORE -> [%292 + 94, %292 + 94 + 64] <- !!!
  …
  STORE -> [%303 + 62, %303 + 62 + 64]

…
KERNEL_END 
EPILOG:

  %299 = PHI %15, %303
  STORE -> [%299 + 94, %299 + 94 + 64]

The new content in [offset 0, offset 0 + 64] in the next iteration is overwritten by the write [offset 94, offset94 + 64] in the previous iteration. The loop carried dependency could also exist between store and store instructions.


Repository:
  rL LLVM

https://reviews.llvm.org/D59036

Files:
  lib/CodeGen/MachinePipeliner.cpp


Index: lib/CodeGen/MachinePipeliner.cpp
===================================================================
--- lib/CodeGen/MachinePipeliner.cpp
+++ lib/CodeGen/MachinePipeliner.cpp
@@ -3131,8 +3131,8 @@
       SI->hasOrderedMemoryRef() || DI->hasOrderedMemoryRef())
     return true;
 
-  // Only chain dependences between a load and store can be loop carried.
-  if (!DI->mayStore() || !SI->mayLoad())
+  if (!(DI->mayStore() && SI->mayLoad()) &&
+      !(DI->mayStore() && SI->mayStore()))
     return false;
 
   unsigned DeltaS, DeltaD;


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D59036.189544.patch
Type: text/x-patch
Size: 542 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190306/269b1a0e/attachment.bin>