[llvm] [LAA] Check if invariant accesses is strictly before other access. (PR #92307)
Florian Hahn via llvm-commits
llvm-commits at lists.llvm.org
Wed May 15 12:41:02 PDT 2024
https://github.com/fhahn created https://github.com/llvm/llvm-project/pull/92307
If we have an invariant access which is strictly before the other access, the accesses are guaranteed to not access any overlapping memory and there is no dependence
If either Src or Sink are loop invariant, check if Src + AccessSize <= Sink (or vice versa). This is done by checking (Sink - (Src + AccessSize)) s>= 0.
>From f5ce2ce91f9badfde3da37cacf9e857a61705097 Mon Sep 17 00:00:00 2001
From: Florian Hahn <flo at fhahn.com>
Date: Thu, 18 Apr 2024 12:07:20 +0100
Subject: [PATCH] [LAA] Check if invariant accesses is strictly before other
access.
If we have an invariant access which is strictly before the other
access, the accesses are guaranteed to not access any overlapping memory
and there is no dependence
If either Src or Sink are loop invariant, check if Src + AccessSize <= Sink
(or vice versa). This is done by checking (Sink - (Src + AccessSize)) s>= 0.
---
llvm/lib/Analysis/LoopAccessAnalysis.cpp | 29 +++++++++--
.../invariant-dependence-before.ll | 49 +++----------------
2 files changed, 33 insertions(+), 45 deletions(-)
diff --git a/llvm/lib/Analysis/LoopAccessAnalysis.cpp b/llvm/lib/Analysis/LoopAccessAnalysis.cpp
index 4ba2e15222106..9e3b15db76b88 100644
--- a/llvm/lib/Analysis/LoopAccessAnalysis.cpp
+++ b/llvm/lib/Analysis/LoopAccessAnalysis.cpp
@@ -1982,6 +1982,30 @@ getDependenceDistanceStrideAndSize(
InnermostLoop))
return MemoryDepChecker::Dependence::IndirectUnsafe;
+ uint64_t TypeByteSizeA = ATy->isScalableTy() ? 0 : DL.getTypeAllocSize(ATy);
+ uint64_t TypeByteSizeB = BTy->isScalableTy() ? 0 : DL.getTypeAllocSize(BTy);
+ // If either Src or Sink are loop invariant, check if Src + AccessSize <= Sink
+ // (or vice versa). This is done by checking (Sink - (Src + AccessSize)) s>=
+ // 0.
+ if (SE.isLoopInvariant(Src, InnermostLoop) && TypeByteSizeA > 0) {
+ const SCEV *SrcLastAccess =
+ SE.getAddExpr(Src, SE.getConstant(Src->getType(), TypeByteSizeA));
+ const SCEV *D = SE.getMinusSCEV(Sink, SrcLastAccess);
+ if (!isa<SCEVCouldNotCompute>(D) &&
+ SE.isKnownPredicate(CmpInst::ICMP_SGE, D,
+ SE.getConstant(Src->getType(), 0)))
+ return MemoryDepChecker::Dependence::NoDep;
+ }
+ if (SE.isLoopInvariant(Sink, InnermostLoop) && TypeByteSizeB > 0) {
+ const SCEV *SinkLastAccess =
+ SE.getAddExpr(Sink, SE.getConstant(Src->getType(), TypeByteSizeB));
+ const SCEV *D = SE.getMinusSCEV(Src, SinkLastAccess);
+ if (!isa<SCEVCouldNotCompute>(D) &&
+ SE.isKnownPredicate(CmpInst::ICMP_SGE, D,
+ SE.getConstant(Src->getType(), 0)))
+ return MemoryDepChecker::Dependence::NoDep;
+ }
+
// Need accesses with constant strides and the same direction. We don't want
// to vectorize "A[B[i]] += ..." and similar code or pointer arithmetic that
// could wrap in the address space.
@@ -1991,13 +2015,12 @@ getDependenceDistanceStrideAndSize(
return MemoryDepChecker::Dependence::Unknown;
}
- uint64_t TypeByteSize = DL.getTypeAllocSize(ATy);
bool HasSameSize =
DL.getTypeStoreSizeInBits(ATy) == DL.getTypeStoreSizeInBits(BTy);
if (!HasSameSize)
- TypeByteSize = 0;
+ TypeByteSizeA = 0;
return DepDistanceStrideAndSizeInfo(Dist, std::abs(StrideAPtr),
- std::abs(StrideBPtr), TypeByteSize,
+ std::abs(StrideBPtr), TypeByteSizeA,
AIsWrite, BIsWrite);
}
diff --git a/llvm/test/Analysis/LoopAccessAnalysis/invariant-dependence-before.ll b/llvm/test/Analysis/LoopAccessAnalysis/invariant-dependence-before.ll
index 2a210a5a445b9..d66d3528586a8 100644
--- a/llvm/test/Analysis/LoopAccessAnalysis/invariant-dependence-before.ll
+++ b/llvm/test/Analysis/LoopAccessAnalysis/invariant-dependence-before.ll
@@ -4,13 +4,8 @@
define void @test_invar_dependence_before_positive_strided_access_1(ptr %a) {
; CHECK-LABEL: 'test_invar_dependence_before_positive_strided_access_1'
; CHECK-NEXT: loop:
-; CHECK-NEXT: Report: unsafe dependent memory operations in loop. Use #pragma clang loop distribute(enable) to allow loop distribution to attempt to isolate the offending operations into a separate loop
-; CHECK-NEXT: Unknown data dependence.
+; CHECK-NEXT: Memory dependences are safe
; CHECK-NEXT: Dependences:
-; CHECK-NEXT: Unknown:
-; CHECK-NEXT: %l = load i32, ptr %a, align 4 ->
-; CHECK-NEXT: store i32 %l, ptr %gep, align 4
-; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-EMPTY:
@@ -39,13 +34,8 @@ exit:
define void @test_invar_dependence_before_positive_strided_access_2(ptr %a) {
; CHECK-LABEL: 'test_invar_dependence_before_positive_strided_access_2'
; CHECK-NEXT: loop:
-; CHECK-NEXT: Report: unsafe dependent memory operations in loop. Use #pragma clang loop distribute(enable) to allow loop distribution to attempt to isolate the offending operations into a separate loop
-; CHECK-NEXT: Unknown data dependence.
+; CHECK-NEXT: Memory dependences are safe
; CHECK-NEXT: Dependences:
-; CHECK-NEXT: Unknown:
-; CHECK-NEXT: %l = load i32, ptr %gep, align 4 ->
-; CHECK-NEXT: store i32 %l, ptr %a, align 4
-; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-EMPTY:
@@ -144,13 +134,8 @@ exit:
define void @test_invar_dependence_before_positive_strided_access_1_different_access_sizes(ptr %a) {
; CHECK-LABEL: 'test_invar_dependence_before_positive_strided_access_1_different_access_sizes'
; CHECK-NEXT: loop:
-; CHECK-NEXT: Report: unsafe dependent memory operations in loop. Use #pragma clang loop distribute(enable) to allow loop distribution to attempt to isolate the offending operations into a separate loop
-; CHECK-NEXT: Unknown data dependence.
+; CHECK-NEXT: Memory dependences are safe
; CHECK-NEXT: Dependences:
-; CHECK-NEXT: Unknown:
-; CHECK-NEXT: %l = load i32, ptr %a, align 4 ->
-; CHECK-NEXT: store i8 %t, ptr %gep, align 1
-; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-EMPTY:
@@ -216,13 +201,8 @@ exit:
define void @test_invar_dependence_before_negative_strided_access_1(ptr %a) {
; CHECK-LABEL: 'test_invar_dependence_before_negative_strided_access_1'
; CHECK-NEXT: loop:
-; CHECK-NEXT: Report: unsafe dependent memory operations in loop. Use #pragma clang loop distribute(enable) to allow loop distribution to attempt to isolate the offending operations into a separate loop
-; CHECK-NEXT: Unknown data dependence.
+; CHECK-NEXT: Memory dependences are safe
; CHECK-NEXT: Dependences:
-; CHECK-NEXT: Unknown:
-; CHECK-NEXT: %l = load i32, ptr %a, align 4 ->
-; CHECK-NEXT: store i32 %l, ptr %gep, align 4
-; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-EMPTY:
@@ -251,13 +231,8 @@ exit:
define void @test_invar_dependence_before_negative_strided_access_2(ptr %a) {
; CHECK-LABEL: 'test_invar_dependence_before_negative_strided_access_2'
; CHECK-NEXT: loop:
-; CHECK-NEXT: Report: unsafe dependent memory operations in loop. Use #pragma clang loop distribute(enable) to allow loop distribution to attempt to isolate the offending operations into a separate loop
-; CHECK-NEXT: Unknown data dependence.
+; CHECK-NEXT: Memory dependences are safe
; CHECK-NEXT: Dependences:
-; CHECK-NEXT: Unknown:
-; CHECK-NEXT: %l = load i32, ptr %gep, align 4 ->
-; CHECK-NEXT: store i32 %l, ptr %a, align 4
-; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-EMPTY:
@@ -357,13 +332,8 @@ exit:
define void @test_both_invar_before_1(ptr %a) {
; CHECK-LABEL: 'test_both_invar_before_1'
; CHECK-NEXT: loop:
-; CHECK-NEXT: Report: unsafe dependent memory operations in loop. Use #pragma clang loop distribute(enable) to allow loop distribution to attempt to isolate the offending operations into a separate loop
-; CHECK-NEXT: Unknown data dependence.
+; CHECK-NEXT: Memory dependences are safe
; CHECK-NEXT: Dependences:
-; CHECK-NEXT: Unknown:
-; CHECK-NEXT: %l = load i32, ptr %a, align 4 ->
-; CHECK-NEXT: store i32 %l, ptr %gep.off, align 4
-; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-EMPTY:
@@ -391,13 +361,8 @@ exit:
define void @test_both_invar_before_2(ptr %a) {
; CHECK-LABEL: 'test_both_invar_before_2'
; CHECK-NEXT: loop:
-; CHECK-NEXT: Report: unsafe dependent memory operations in loop. Use #pragma clang loop distribute(enable) to allow loop distribution to attempt to isolate the offending operations into a separate loop
-; CHECK-NEXT: Unknown data dependence.
+; CHECK-NEXT: Memory dependences are safe
; CHECK-NEXT: Dependences:
-; CHECK-NEXT: Unknown:
-; CHECK-NEXT: %l = load i32, ptr %gep.off, align 4 ->
-; CHECK-NEXT: store i32 %l, ptr %a, align 4
-; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-EMPTY:
More information about the llvm-commits
mailing list