[llvm] [LAA] Refine stride checks for SCEVs during dependence analysis. (PR #99577)
Michael Kruse via llvm-commits
llvm-commits at lists.llvm.org
Thu Jul 25 04:13:27 PDT 2024
================
@@ -1987,11 +1979,47 @@ MemoryDepChecker::getDependenceDistanceStrideAndSize(
}
}
- // Need accesses with constant strides and the same direction. We don't want
- // to vectorize "A[B[i]] += ..." and similar code or pointer arithmetic that
- // could wrap in the address space.
- if (!StrideAPtr || !StrideBPtr || (StrideAPtr > 0 && StrideBPtr < 0) ||
- (StrideAPtr < 0 && StrideBPtr > 0)) {
+ // Need accesses with constant strides and the same direction for further
+ // dependence analysis. We don't want to vectorize "A[B[i]] += ..." and
+ // similar code or pointer arithmetic that could wrap in the address space.
+ //
+ // If Src or Sink are non-wrapping AddRecs, StrideAPtr and StrideBPtr contain
+ // a SCEV representing the stride of the SCEV. It may not be a known constant
+ // value though.
+
+ // If either Src or Sink are not strided (i.e. not a non-wrapping AddRec), we
+ // cannot analyze the dependence further.
+ if (!StrideAPtr || !StrideBPtr) {
+ bool SrcInvariant = SE.isLoopInvariant(Src, InnermostLoop);
+ bool SinkInvariant = SE.isLoopInvariant(Sink, InnermostLoop);
+ // If either Src or Sink are not loop invariant and not strided, the
+ // expression in the current iteration may overlap with any earlier or later
+ // iteration. This is not safe and we also cannot generate runtime checks to
+ // ensure safety. This includes expressions where an index is loaded in each
+ // iteration or wrapping AddRecs.
+ if ((!SrcInvariant && !StrideAPtr) || (!SinkInvariant && !StrideBPtr))
+ return MemoryDepChecker::Dependence::IndirectUnsafe;
+
+ // Otherwise both Src or Sink are either loop invariant or strided and we
+ // can generate a runtime check to disambiguate the accesses.
+ return MemoryDepChecker::Dependence::Unknown;
+ }
+
+ LLVM_DEBUG(dbgs() << "LAA: Src induction step: " << **StrideAPtr
+ << " Sink induction step: " << **StrideBPtr << "\n");
+ // If either Src or Sink have a non-constant stride, we can generate a runtime
+ // check to disambiguate them.
+ if ((!isa<SCEVConstant>(*StrideAPtr)) || (!isa<SCEVConstant>(*StrideBPtr)))
----------------
Meinersbur wrote:
```suggestion
if (!isa<SCEVConstant>(*StrideAPtr) || !isa<SCEVConstant>(*StrideBPtr))
```
`!` has a higher precedence than `||` so we can save some parens.
However, `getPtrStrideSCEV` only ever returns `SCEVConstant` (or `nullopt`), so I don't get why making this patch more complicated by including 1debdc19f914b699db8f1ab0225da6e1bc61583a.
https://github.com/llvm/llvm-project/pull/99577
More information about the llvm-commits
mailing list