[PATCH] D68395: [LoopDataPrefetch] Update an existing prefetch to same address to 'write' for a store

[Jonas Paulsson via Phabricator via llvm-commits]

jonpa created this revision.
jonpa added reviewers: uweigand, hfinkel, anemet, jfb.
Herald added a subscriber: dexonsmith.

If a previous read access in the loop has gotten a prefetch, that prefetch should be updated to 'write' in case a store is later seen to the same address.

I believe one should be careful not to emit store prefetches without relative certainty of benefit (targeted cache line), so this patch only does this in case the address is actually the same (PD == 0). I however also tried to do this with different spaces between the load and the store. On SystemZ (with '-min-prefetch-stride=128 -loop-prefetch-writes') I see

(unpatched) 1606 read / 601 write prefetches.
patch (same address): 1503 / 704.
patch, but within Cache Line Size: 1421 / 786
patch, but within CLS / 16: 1459 / 748

In summary, this seems to be worthwhile and safe even for just the identical address case. Perhaps CLS/16 is beneficial..?


https://reviews.llvm.org/D68395

Files:
  lib/Transforms/Scalar/LoopDataPrefetch.cpp
  test/CodeGen/SystemZ/prefetch-04.ll


Index: test/CodeGen/SystemZ/prefetch-04.ll
===================================================================
--- /dev/null
+++ test/CodeGen/SystemZ/prefetch-04.ll
@@ -0,0 +1,28 @@
+; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z14 -prefetch-distance=50 \
+; RUN:   -loop-prefetch-writes -stop-after=loop-data-prefetch | FileCheck %s
+;
+; Check that for a load followed by a store to the same address gets a single
+; write prefetch.
+;
+; CHECK-LABEL: for.body
+; CHECK: call void @llvm.prefetch.p0i8(i8* %scevgep{{.*}}, i32 1, i32 3, i32 1
+; CHECK-not: call void @llvm.prefetch
+
+define void @fun(i32* nocapture %Src, i32* nocapture readonly %Dst) {
+entry:
+  br label %for.body
+
+for.body:
+  %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next.9, %for.body ]
+  %arrayidx = getelementptr inbounds i32, i32* %Dst, i64 %indvars.iv
+  %0 = load i32, i32* %arrayidx, align 4
+  %a = add i32 %0, 128
+  store i32 %a, i32* %arrayidx, align 4
+  %indvars.iv.next.9 = add nuw nsw i64 %indvars.iv, 1600
+  %cmp.9 = icmp ult i64 %indvars.iv.next.9, 11200
+  br i1 %cmp.9, label %for.body, label %for.cond.cleanup
+
+for.cond.cleanup:
+  ret void
+}
+
Index: lib/Transforms/Scalar/LoopDataPrefetch.cpp
===================================================================
--- lib/Transforms/Scalar/LoopDataPrefetch.cpp
+++ lib/Transforms/Scalar/LoopDataPrefetch.cpp
@@ -313,6 +313,14 @@
               if (DomBB != PrefetchBB)
                 Pref.first->moveBefore(DomBB->getTerminator());
             }
+            /// If MemI is a store but the already emitted prefetch is for a
+            /// 'read', update the prefetch argument to 'write'.
+            bool HasLoadPrefetch =
+              cast<ConstantInt>(Pref.first->getOperand(1))->getZExtValue() == 0;
+            if (PD == 0 && HasLoadPrefetch && MemI->mayWriteToMemory()) {
+              Type *I32 = Type::getInt32Ty(BB->getContext());
+              Pref.first->setOperand(1, ConstantInt::get(I32, 1));
+            }
             DupPref = true;
             break;
           }


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D68395.223011.patch
Type: text/x-patch
Size: 2054 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20191003/89336d98/attachment.bin>