[llvm] [LoopIdiom] Perform loop versioning to use memcpy (PR #125043)

Ryotaro Kasuga via llvm-commits llvm-commits at lists.llvm.org
Thu Jan 30 01:59:20 PST 2025


================
@@ -1486,6 +1514,83 @@ bool LoopIdiomRecognize::avoidLIRForMultiBlockLoop(bool IsMemset,
   return false;
 }
 
+// Returns true if we should version the loop and make sure that there is no
+// alias between the store and the load. This allows us to use `memcpy` instead
+// of `memmove`. However, versioning increases the code size. In the worst case,
+// if there are multiple load/store pairs, the code size increases
+// exponentially. Therefore, versioning is supported only if the loop only does
+// transfers related to this store and load. That is, we will version the loop
+// as follows:
+//
+//   ```
+//   for (i=0; i<len; i++)
+//     dst[i] = src[i];
+//   ```
+//
+// But we don't want to do this if there are other processes inside the loop,
+// e.g.,
+//
+//   ```
+//   acc = 0;
+//   for (i=0; i<len; i++) {
+//     dst[i] = src[i];
+//     acc += ...;
+//   }
+//   ```
+bool LoopIdiomRecognize::shouldVersionLoopForMemCpy(
+    Instruction *TheStore, Instruction *TheLoad) const {
+  if (ApplyCodeSizeHeuristics || !EnableLoopVersioning)
+    return false;
+
+  // There are cases where the load and store always overlap. Avoid versioning
+  // in these situations.
+  auto *Checking = LAI.getRuntimePointerChecking();
+  if (Checking->getNumberOfChecks() == 0)
+    return false;
+
+  BasicBlock *Cur = TheStore->getParent();
+  for (auto &I : *Cur) {
+    if (I.isDebugOrPseudoInst() || I.isTerminator())
+      continue;
+
+    // If there is a memory instruction other then `TheStore` and `TheLoad`,
+    // then bail out.
+    if (I.mayReadOrWriteMemory() && (&I) != TheStore && (&I) != TheLoad)
+      return false;
+
+    // We also abandon the versioning if there is an instruction other than
+    // `TheStore`, `TheLoad`, and anything related to loop control.
+    for (const auto &U : I.uses()) {
+      const Instruction *UseI = cast<Instruction>(U.getUser());
+      if (UseI->getParent() != Cur)
+        return false;
+    }
+  }
----------------
kasuga-fj wrote:

I'm not sure if this process is achieving what I want to do. I'd like to target the loop that is "dedicated" to the copy from `TheLoad` to `TheStore`.

https://github.com/llvm/llvm-project/pull/125043


More information about the llvm-commits mailing list