[llvm] [LAA] Support different strides & non constant dep distances using SCEV. (PR #88039)

Florian Hahn via llvm-commits llvm-commits at lists.llvm.org
Mon Apr 22 08:48:31 PDT 2024


================
@@ -2071,34 +2092,51 @@ MemoryDepChecker::Dependence::DepType MemoryDepChecker::isDependent(
     // NOTE: There is no need to update MaxSafeVectorWidthInBits after call to
     // couldPreventStoreLoadForward, even if it changed MinDepDistBytes, since a
     // forward dependency will allow vectorization using any width.
-    if (IsTrueDataDependence && EnableForwardingConflictDetection &&
-        (!HasSameSize || couldPreventStoreLoadForward(Val.abs().getZExtValue(),
-                                                      TypeByteSize))) {
-      LLVM_DEBUG(dbgs() << "LAA: Forward but may prevent st->ld forwarding\n");
-      return Dependence::ForwardButPreventsForwarding;
+    if (IsTrueDataDependence && EnableForwardingConflictDetection) {
+      if (!C) {
+        // TODO: Relax requirement that there is a common stride to retry with
+        // non-constant distance dependencies.
+        FoundNonConstantDistanceDependence |= !!CommonStride;
+        return Dependence::Unknown;
+      }
+      if (!HasSameSize ||
+          couldPreventStoreLoadForward(C->getAPInt().abs().getZExtValue(),
+                                       TypeByteSize)) {
+        LLVM_DEBUG(
+            dbgs() << "LAA: Forward but may prevent st->ld forwarding\n");
+        return Dependence::ForwardButPreventsForwarding;
+      }
     }
 
     LLVM_DEBUG(dbgs() << "LAA: Dependence is negative\n");
     return Dependence::Forward;
   }
 
-  // Write to the same location with the same size.
-  if (Val == 0) {
-    if (HasSameSize)
-      return Dependence::Forward;
-    LLVM_DEBUG(
-        dbgs() << "LAA: Zero dependence difference but different type sizes\n");
+  if (!C) {
+    // TODO: Relax requirement that there is a common stride to retry with
+    // non-constant distance dependencies.
+    FoundNonConstantDistanceDependence |= !!CommonStride;
+    LLVM_DEBUG(dbgs() << "LAA: Dependence because of non-constant distance\n");
----------------
fhahn wrote:

It is independent of whether there is a common stride or not, but previously `getDependenceDistanceStrideAndSize`  would exit early with `Unknown` if the strides weren't matching, so `FoundNonConstantDistanceDependence` was only ever set if there was a common stride which is what the current patch tries to preserve in the initial version.

The current `FoundNonConstantDistanceDependence` can lead to runtime check generation in cases where the runtime checks will always fail unfortunately, which is why I'd like to keep its original behavior for now, as we regress codegen otherwise in some cases.

https://github.com/llvm/llvm-project/pull/88039


More information about the llvm-commits mailing list