[polly] r302636 - [Polly] Canonicalize arrays according to base-ptr equivalence class
Michael Kruse via llvm-commits
llvm-commits at lists.llvm.org
Tue May 30 05:26:12 PDT 2017
Just a though why I had concerns using setNewAccessRelation for this purpose.
setNewAccessRelation will trigger code-generation of the index
expression. This is not possible if the index expression is affine.
That is, this patch effectively breaks load-hoisting of base pointers
that have one or more non-affine access.
Michael
2017-05-10 12:59 GMT+02:00 Tobias Grosser via llvm-commits
<llvm-commits at lists.llvm.org>:
> Author: grosser
> Date: Wed May 10 05:59:58 2017
> New Revision: 302636
>
> URL: http://llvm.org/viewvc/llvm-project?rev=302636&view=rev
> Log:
> [Polly] Canonicalize arrays according to base-ptr equivalence class
>
> Summary:
> In case two arrays share base pointers in the same invariant load equivalence
> class, we canonicalize all memory accesses to the first of these arrays
> (according to their order in the equivalence class).
>
> This enables us to optimize kernels such as boost::ublas by ensuring that
> different references to the C array are interpreted as accesses to the same
> array. Before this change the runtime alias check for ublas would fail, as it
> would assume models of the C array with differing (but identically valued) base
> pointers would reference distinct regions of memory whereas the referenced
> memory regions were indeed identical.
>
> As part of this change we remove most of the MemoryAccess::get*BaseAddr
> interface. We removed already all references to get*BaseAddr in previous
> commits to ensure that no code relies on matching base pointers between
> memory accesses and scop arrays -- except for three remaining uses where we
> need the original base pointer. We document for these situations that
> MemoryAccess::getOriginalBaseAddr may return a base pointer that is distinct
> to the base pointer of the scop array referenced by this memory access.
>
> Reviewers: sebpop, Meinersbur, zinob, gareevroman, pollydev, huihuiz, efriedma, jdoerfert
>
> Reviewed By: Meinersbur
>
> Subscribers: etherzhhb
>
> Tags: #polly
>
> Differential Revision: https://reviews.llvm.org/D28518
>
> Added:
> polly/trunk/test/Isl/CodeGen/invariant_load_canonicalize_array_baseptrs.ll
> polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs.ll
> polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_2.ll
> polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_3.ll
> polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4.ll
> polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4b.ll
> polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4c.ll
> polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_5.ll
> Modified:
> polly/trunk/include/polly/ScopInfo.h
> polly/trunk/lib/Analysis/ScopBuilder.cpp
> polly/trunk/lib/Analysis/ScopInfo.cpp
> polly/trunk/lib/CodeGen/IslNodeBuilder.cpp
> polly/trunk/lib/Transform/ScheduleOptimizer.cpp
>
> Modified: polly/trunk/include/polly/ScopInfo.h
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/include/polly/ScopInfo.h?rev=302636&r1=302635&r2=302636&view=diff
> ==============================================================================
> --- polly/trunk/include/polly/ScopInfo.h (original)
> +++ polly/trunk/include/polly/ScopInfo.h Wed May 10 05:59:58 2017
> @@ -367,6 +367,16 @@ public:
> /// If the array is read only
> bool isReadOnly();
>
> + /// Verify that @p Array is compatible to this ScopArrayInfo.
> + ///
> + /// Two arrays are compatible if their dimensionality, the sizes of their
> + /// dimensions, and their element sizes match.
> + ///
> + /// @param Array The array to compare against.
> + ///
> + /// @returns True, if the arrays are compatible, False otherwise.
> + bool isCompatibleWith(const ScopArrayInfo *Array) const;
> +
> private:
> void addDerivedSAI(ScopArrayInfo *DerivedSAI) {
> DerivedSAIs.insert(DerivedSAI);
> @@ -804,22 +814,14 @@ public:
> /// Get an isl string representing a new access function, if available.
> std::string getNewAccessRelationStr() const;
>
> - /// Get the base address of this access (e.g. A for A[i+j]) when
> + /// Get the original base address of this access (e.g. A for A[i+j]) when
> /// detected.
> - Value *getOriginalBaseAddr() const {
> - assert(!getOriginalScopArrayInfo() /* may noy yet be initialized */ ||
> - getOriginalScopArrayInfo()->getBasePtr() == BaseAddr);
> - return BaseAddr;
> - }
> -
> - /// Get the base address of this access (e.g. A for A[i+j]) after a
> - /// potential change by setNewAccessRelation().
> - Value *getLatestBaseAddr() const {
> - return getLatestScopArrayInfo()->getBasePtr();
> - }
> -
> - /// Old name for getOriginalBaseAddr().
> - Value *getBaseAddr() const { return getOriginalBaseAddr(); }
> + ///
> + /// This adress may differ from the base address referenced by the Original
> + /// ScopArrayInfo to which this array belongs, as this memory access may
> + /// have been unified to a ScopArray which has a different but identically
> + /// valued base pointer in case invariant load hoisting is enabled.
> + Value *getOriginalBaseAddr() const { return BaseAddr; }
>
> /// Get the detection-time base array isl_id for this access.
> __isl_give isl_id *getOriginalArrayId() const;
> @@ -1890,6 +1892,34 @@ private:
> ///
> void hoistInvariantLoads();
>
> + /// Canonicalize arrays with base pointers from the same equivalence class.
> + ///
> + /// Some context: in our normal model we assume that each base pointer is
> + /// related to a single specific memory region, where memory regions
> + /// associated with different base pointers are disjoint. Consequently we do
> + /// not need to compute additional data dependences that model possible
> + /// overlaps of these memory regions. To verify our assumption we compute
> + /// alias checks that verify that modeled arrays indeed do not overlap. In
> + /// case an overlap is detected the runtime check fails and we fall back to
> + /// the original code.
> + ///
> + /// In case of arrays where the base pointers are know to be identical,
> + /// because they are dynamically loaded by accesses that are in the same
> + /// invariant load equivalence class, such run-time alias check would always
> + /// be false.
> + ///
> + /// This function makes sure that we do not generate consistently failing
> + /// run-time checks for code that contains distinct arrays with known
> + /// equivalent base pointers. It identifies for each invariant load
> + /// equivalence class a single canonical array and canonicalizes all memory
> + /// accesses that reference arrays that have base pointers that are known to
> + /// be equal to the base pointer of such a canonical array to this canonical
> + /// array.
> + ///
> + /// We currently do not canonicalize arrays for which certain memory accesses
> + /// have been hoisted as loop invariant.
> + void canonicalizeDynamicBasePtrs();
> +
> /// Add invariant loads listed in @p InvMAs with the domain of @p Stmt.
> void addInvariantLoads(ScopStmt &Stmt, InvariantAccessesTy &InvMAs);
>
> @@ -2483,6 +2513,18 @@ public:
> ///
> /// @param BasePtr The base pointer the object has been stored for.
> /// @param Kind The kind of array info object.
> + ///
> + /// @returns The ScopArrayInfo pointer or NULL if no such pointer is
> + /// available.
> + const ScopArrayInfo *getScopArrayInfoOrNull(Value *BasePtr, MemoryKind Kind);
> +
> + /// Return the cached ScopArrayInfo object for @p BasePtr.
> + ///
> + /// @param BasePtr The base pointer the object has been stored for.
> + /// @param Kind The kind of array info object.
> + ///
> + /// @returns The ScopArrayInfo pointer (may assert if no such pointer is
> + /// available).
> const ScopArrayInfo *getScopArrayInfo(Value *BasePtr, MemoryKind Kind);
>
> /// Invalidate ScopArrayInfo object for base address.
>
> Modified: polly/trunk/lib/Analysis/ScopBuilder.cpp
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/lib/Analysis/ScopBuilder.cpp?rev=302636&r1=302635&r2=302636&view=diff
> ==============================================================================
> --- polly/trunk/lib/Analysis/ScopBuilder.cpp (original)
> +++ polly/trunk/lib/Analysis/ScopBuilder.cpp Wed May 10 05:59:58 2017
> @@ -776,6 +776,7 @@ void ScopBuilder::buildScop(Region &R, A
> return;
>
> scop->hoistInvariantLoads();
> + scop->canonicalizeDynamicBasePtrs();
> scop->verifyInvariantLoads();
> scop->simplifySCoP(true);
>
>
> Modified: polly/trunk/lib/Analysis/ScopInfo.cpp
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/lib/Analysis/ScopInfo.cpp?rev=302636&r1=302635&r2=302636&view=diff
> ==============================================================================
> --- polly/trunk/lib/Analysis/ScopInfo.cpp (original)
> +++ polly/trunk/lib/Analysis/ScopInfo.cpp Wed May 10 05:59:58 2017
> @@ -279,6 +279,20 @@ bool ScopArrayInfo::isReadOnly() {
> return IsReadOnly;
> }
>
> +bool ScopArrayInfo::isCompatibleWith(const ScopArrayInfo *Array) const {
> + if (Array->getElementType() != getElementType())
> + return false;
> +
> + if (Array->getNumberOfDimensions() != getNumberOfDimensions())
> + return false;
> +
> + for (unsigned i = 0; i < getNumberOfDimensions(); i++)
> + if (Array->getDimensionSize(i) != getDimensionSize(i))
> + return false;
> +
> + return true;
> +}
> +
> void ScopArrayInfo::updateElementType(Type *NewElementType) {
> if (NewElementType == ElementType)
> return;
> @@ -3765,6 +3779,76 @@ void Scop::hoistInvariantLoads() {
> isl_union_map_free(Writes);
> }
>
> +/// Find the canonical scop array info object for a set of invariant load
> +/// hoisted loads. The canonical array is the one that corresponds to the
> +/// first load in the list of accesses which is used as base pointer of a
> +/// scop array.
> +static const ScopArrayInfo *findCanonicalArray(Scop *S,
> + MemoryAccessList &Accesses) {
> + for (MemoryAccess *Access : Accesses) {
> + const ScopArrayInfo *CanonicalArray = S->getScopArrayInfoOrNull(
> + Access->getAccessInstruction(), MemoryKind::Array);
> + if (CanonicalArray)
> + return CanonicalArray;
> + }
> + return nullptr;
> +}
> +
> +/// Check if @p Array severs as base array in an invariant load.
> +static bool isUsedForIndirectHoistedLoad(Scop *S, const ScopArrayInfo *Array) {
> + for (InvariantEquivClassTy &EqClass2 : S->getInvariantAccesses())
> + for (MemoryAccess *Access2 : EqClass2.InvariantAccesses)
> + if (Access2->getScopArrayInfo() == Array)
> + return true;
> + return false;
> +}
> +
> +/// Replace the base pointer arrays in all memory accesses referencing @p Old,
> +/// with a reference to @p New.
> +static void replaceBasePtrArrays(Scop *S, const ScopArrayInfo *Old,
> + const ScopArrayInfo *New) {
> + for (ScopStmt &Stmt : *S)
> + for (MemoryAccess *Access : Stmt) {
> + if (Access->getLatestScopArrayInfo() != Old)
> + continue;
> +
> + isl_id *Id = New->getBasePtrId();
> + isl_map *Map = Access->getAccessRelation();
> + Map = isl_map_set_tuple_id(Map, isl_dim_out, Id);
> + Access->setAccessRelation(Map);
> + }
> +}
> +
> +void Scop::canonicalizeDynamicBasePtrs() {
> + for (InvariantEquivClassTy &EqClass : InvariantEquivClasses) {
> + MemoryAccessList &BasePtrAccesses = EqClass.InvariantAccesses;
> +
> + const ScopArrayInfo *CanonicalBasePtrSAI =
> + findCanonicalArray(this, BasePtrAccesses);
> +
> + if (!CanonicalBasePtrSAI)
> + continue;
> +
> + for (MemoryAccess *BasePtrAccess : BasePtrAccesses) {
> + const ScopArrayInfo *BasePtrSAI = getScopArrayInfoOrNull(
> + BasePtrAccess->getAccessInstruction(), MemoryKind::Array);
> + if (!BasePtrSAI || BasePtrSAI == CanonicalBasePtrSAI ||
> + !BasePtrSAI->isCompatibleWith(CanonicalBasePtrSAI))
> + continue;
> +
> + // we currently do not canonicalize arrays where some accesses are
> + // hoisted as invariant loads. If we would, we need to update the access
> + // function of the invariant loads as well. However, as this is not a
> + // very common situation, we leave this for now to avoid further
> + // complexity increases.
> + if (isUsedForIndirectHoistedLoad(this, BasePtrSAI))
> + continue;
> +
> + replaceBasePtrArrays(this, BasePtrSAI, CanonicalBasePtrSAI);
> + }
> + }
> +}
> +
> const ScopArrayInfo *
> Scop::getOrCreateScopArrayInfo(Value *BasePtr, Type *ElementType,
> ArrayRef<const SCEV *> Sizes, MemoryKind Kind,
> @@ -3806,8 +3890,14 @@ Scop::createScopArrayInfo(Type *ElementT
> return SAI;
> }
>
> -const ScopArrayInfo *Scop::getScopArrayInfo(Value *BasePtr, MemoryKind Kind) {
> +const ScopArrayInfo *Scop::getScopArrayInfoOrNull(Value *BasePtr,
> + MemoryKind Kind) {
> auto *SAI = ScopArrayInfoMap[std::make_pair(BasePtr, Kind)].get();
> + return SAI;
> +}
> +
> +const ScopArrayInfo *Scop::getScopArrayInfo(Value *BasePtr, MemoryKind Kind) {
> + auto *SAI = getScopArrayInfoOrNull(BasePtr, Kind);
> assert(SAI && "No ScopArrayInfo available for this base pointer");
> return SAI;
> }
>
> Modified: polly/trunk/lib/CodeGen/IslNodeBuilder.cpp
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/lib/CodeGen/IslNodeBuilder.cpp?rev=302636&r1=302635&r2=302636&view=diff
> ==============================================================================
> --- polly/trunk/lib/CodeGen/IslNodeBuilder.cpp (original)
> +++ polly/trunk/lib/CodeGen/IslNodeBuilder.cpp Wed May 10 05:59:58 2017
> @@ -1229,7 +1229,7 @@ bool IslNodeBuilder::preloadInvariantEqu
> // current SAI could be the base pointer of the derived SAI, however we
> // should only change the base pointer of the derived SAI if we actually
> // preloaded it.
> - if (BasePtr == MA->getBaseAddr()) {
> + if (BasePtr == MA->getOriginalBaseAddr()) {
> assert(BasePtr->getType() == PreloadVal->getType());
> DerivedSAI->setBasePtr(PreloadVal);
> }
>
> Modified: polly/trunk/lib/Transform/ScheduleOptimizer.cpp
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/lib/Transform/ScheduleOptimizer.cpp?rev=302636&r1=302635&r2=302636&view=diff
> ==============================================================================
> --- polly/trunk/lib/Transform/ScheduleOptimizer.cpp (original)
> +++ polly/trunk/lib/Transform/ScheduleOptimizer.cpp Wed May 10 05:59:58 2017
> @@ -1304,7 +1304,8 @@ __isl_give isl_schedule_node *ScheduleTr
> __isl_take isl_schedule_node *Node, const llvm::TargetTransformInfo *TTI,
> MatMulInfoTy &MMI) {
> assert(TTI && "The target transform info should be provided.");
> - Node = markInterIterationAliasFree(Node, MMI.WriteToC->getLatestBaseAddr());
> + Node = markInterIterationAliasFree(
> + Node, MMI.WriteToC->getLatestScopArrayInfo()->getBasePtr());
> int DimOutNum = isl_schedule_node_band_n_member(Node);
> assert(DimOutNum > 2 && "In case of the matrix multiplication the loop nest "
> "and, consequently, the corresponding scheduling "
>
> Added: polly/trunk/test/Isl/CodeGen/invariant_load_canonicalize_array_baseptrs.ll
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/test/Isl/CodeGen/invariant_load_canonicalize_array_baseptrs.ll?rev=302636&view=auto
> ==============================================================================
> --- polly/trunk/test/Isl/CodeGen/invariant_load_canonicalize_array_baseptrs.ll (added)
> +++ polly/trunk/test/Isl/CodeGen/invariant_load_canonicalize_array_baseptrs.ll Wed May 10 05:59:58 2017
> @@ -0,0 +1,36 @@
> +; RUN: opt %loadPolly -polly-codegen -S < %s \
> +; RUN: -polly-invariant-load-hoisting \
> +; RUN: | FileCheck %s
> +
> +; CHECK: %polly.access.A = getelementptr float*, float** %A, i64 0
> +; CHECK: %polly.access.A.load = load float*, float** %polly.access.A
> +; CHECK: store float 4.200000e+01, float* %polly.access.A.load
> +; CHECK: store float 4.800000e+01, float* %polly.access.A.load
> +
> +define void @foo(float** %A) {
> +start:
> + br label %loop
> +
> +loop:
> + %indvar = phi i64 [0, %start], [%indvar.next, %latch]
> + %indvar.next = add nsw i64 %indvar, 1
> + %icmp = icmp slt i64 %indvar.next, 1024
> + br i1 %icmp, label %body1, label %exit
> +
> +body1:
> + %baseA = load float*, float** %A
> + store float 42.0, float* %baseA
> + br label %body2
> +
> +body2:
> + %baseB = load float*, float** %A
> + store float 48.0, float* %baseB
> + br label %latch
> +
> +latch:
> + br label %loop
> +
> +exit:
> + ret void
> +
> +}
>
> Added: polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs.ll
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs.ll?rev=302636&view=auto
> ==============================================================================
> --- polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs.ll (added)
> +++ polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs.ll Wed May 10 05:59:58 2017
> @@ -0,0 +1,46 @@
> +; RUN: opt %loadPolly -polly-scops -analyze < %s \
> +; RUN: -polly-invariant-load-hoisting \
> +; RUN: | FileCheck %s
> +
> +; CHECK: Stmt_body1
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body1[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body1[i0] -> [i0, 0] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body1[i0] -> MemRef_baseB[0] };
> +; CHECK-NEXT: Stmt_body2
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body2[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body2[i0] -> [i0, 1] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body2[i0] -> MemRef_baseB[0] };
> +
> +define void @foo(float** %A) {
> +start:
> + br label %loop
> +
> +loop:
> + %indvar = phi i64 [0, %start], [%indvar.next, %latch]
> + %indvar.next = add nsw i64 %indvar, 1
> + %icmp = icmp slt i64 %indvar.next, 1024
> + br i1 %icmp, label %body1, label %exit
> +
> +body1:
> + %baseA = load float*, float** %A
> + store float 42.0, float* %baseA
> + br label %body2
> +
> +body2:
> + %baseB = load float*, float** %A
> + store float 42.0, float* %baseB
> + br label %latch
> +
> +latch:
> + br label %loop
> +
> +exit:
> + ret void
> +
> +}
>
> Added: polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_2.ll
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_2.ll?rev=302636&view=auto
> ==============================================================================
> --- polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_2.ll (added)
> +++ polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_2.ll Wed May 10 05:59:58 2017
> @@ -0,0 +1,91 @@
> +; RUN: opt %loadPolly -polly-scops -analyze < %s \
> +; RUN: -polly-invariant-load-hoisting \
> +; RUN: | FileCheck %s
> +
> +; Make sure we choose a canonical element that is not the first invariant load,
> +; but the first that is an array base pointer.
> +
> +; CHECK: Statements {
> +; CHECK-NEXT: Stmt_body0
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body0[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body0[i0] -> [i0, 0] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body0[i0] -> MemRef_X[0] };
> +; CHECK-NEXT: Stmt_body1
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body1[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body1[i0] -> [i0, 1] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body1[i0] -> MemRef_baseB[0] };
> +; CHECK-NEXT: Stmt_body2
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body2[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body2[i0] -> [i0, 2] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body2[i0] -> MemRef_X[0] };
> +; CHECK-NEXT: ReadAccess := [Reduction Type: NONE] [Scalar: 1]
> +; CHECK-NEXT: { Stmt_body2[i0] -> MemRef_ptr[] };
> +; CHECK-NEXT: Stmt_body3
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body3[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body3[i0] -> [i0, 3] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body3[i0] -> MemRef_baseB[0] };
> +; CHECK-NEXT: Stmt_body4
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body4[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body4[i0] -> [i0, 4] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body4[i0] -> MemRef_X[0] };
> +; CHECK-NEXT: ReadAccess := [Reduction Type: NONE] [Scalar: 1]
> +; CHECK-NEXT: { Stmt_body4[i0] -> MemRef_ptr[] };
> +; CHECK-NEXT: }
> +
> +define void @foo(float** %A, float** %X) {
> +start:
> + br label %loop
> +
> +loop:
> + %indvar = phi i64 [0, %start], [%indvar.next, %latch]
> + %indvar.next = add nsw i64 %indvar, 1
> + %icmp = icmp slt i64 %indvar.next, 1024
> + br i1 %icmp, label %body0, label %exit
> +
> +body0:
> + %ptr = load float*, float** %A
> + store float* %ptr, float** %X
> + br label %body1
> +
> +body1:
> + %baseA = load float*, float** %A
> + store float 42.0, float* %baseA
> + br label %body2
> +
> +body2:
> + %ptr2 = load float*, float** %A
> + store float* %ptr, float** %X
> + br label %body3
> +
> +body3:
> + %baseB = load float*, float** %A
> + store float 42.0, float* %baseB
> + br label %body4
> +
> +body4:
> + %ptr3 = load float*, float** %A
> + store float* %ptr, float** %X
> + br label %latch
> +
> +latch:
> + br label %loop
> +
> +exit:
> + ret void
> +
> +}
>
> Added: polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_3.ll
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_3.ll?rev=302636&view=auto
> ==============================================================================
> --- polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_3.ll (added)
> +++ polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_3.ll Wed May 10 05:59:58 2017
> @@ -0,0 +1,58 @@
> +; RUN: opt %loadPolly -polly-scops -analyze < %s \
> +; RUN: -polly-invariant-load-hoisting \
> +; RUN: | FileCheck %s
> +
> +; Verify that we canonicalize accesses even tough one of the accesses (even
> +; the canonical base) has a partial execution context. This is correct as
> +; the combined execution context still coveres both accesses.
> +
> +; CHECK: Invariant Accesses: {
> +; CHECK-NEXT: ReadAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body2[i0] -> MemRef_A[0] };
> +; CHECK-NEXT: Execution Context: { : }
> +; CHECK-NEXT: }
> +
> +; CHECK: Stmt_body1
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body1[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body1[i0] -> [i0, 0] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body1[i0] -> MemRef_baseB[0] };
> +; CHECK-NEXT: Stmt_body2
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body2[i0] : 0 <= i0 <= 510 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body2[i0] -> [i0, 1] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body2[i0] -> MemRef_baseB[0] };
> +
> +
> +define void @foo(float** %A) {
> +start:
> + br label %loop
> +
> +loop:
> + %indvar = phi i64 [0, %start], [%indvar.next, %latch]
> + %indvar.next = add nsw i64 %indvar, 1
> + %icmp = icmp slt i64 %indvar.next, 1024
> + br i1 %icmp, label %body1, label %exit
> +
> +body1:
> + %baseA = load float*, float** %A
> + store float 42.0, float* %baseA
> + %cmp = icmp slt i64 %indvar.next, 512
> + br i1 %cmp, label %body2, label %latch
> +
> +body2:
> + %baseB = load float*, float** %A
> + store float 42.0, float* %baseB
> + br label %latch
> +
> +latch:
> + br label %loop
> +
> +exit:
> + ret void
> +
> +}
>
> Added: polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4.ll
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4.ll?rev=302636&view=auto
> ==============================================================================
> --- polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4.ll (added)
> +++ polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4.ll Wed May 10 05:59:58 2017
> @@ -0,0 +1,54 @@
> +; RUN: opt %loadPolly -polly-scops -analyze < %s \
> +; RUN: -polly-invariant-load-hoisting \
> +; RUN: | FileCheck %s
> +
> +; Verify that a delinearized and a not delinearized access are not
> +; canonizalized.
> +
> +; CHECK: Stmt_body1
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: [n] -> { Stmt_body1[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: [n] -> { Stmt_body1[i0] -> [i0, 0] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: [n] -> { Stmt_body1[i0] -> MemRef_baseB[0] };
> +; CHECK-NEXT: Stmt_body2
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: [n] -> { Stmt_body2[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: [n] -> { Stmt_body2[i0] -> [i0, 1] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: [n] -> { Stmt_body2[i0] -> MemRef_baseA[i0, i0] };
> +; CHECK-NEXT: }
> +
> +
> +define void @foo(float** %A, i64 %n, i64 %m) {
> +start:
> + br label %loop
> +
> +loop:
> + %indvar = phi i64 [0, %start], [%indvar.next, %latch]
> + %indvar.next = add nsw i64 %indvar, 1
> + %icmp = icmp slt i64 %indvar.next, 1024
> + br i1 %icmp, label %body1, label %exit
> +
> +body1:
> + %baseB = load float*, float** %A
> + store float 42.0, float* %baseB
> + br label %body2
> +
> +body2:
> + %baseA = load float*, float** %A
> + %offsetA = mul i64 %indvar, %n
> + %offsetA2 = add i64 %offsetA, %indvar
> + %ptrA = getelementptr float, float* %baseA, i64 %offsetA2
> + store float 42.0, float* %ptrA
> + br label %latch
> +
> +latch:
> + br label %loop
> +
> +exit:
> + ret void
> +
> +}
>
> Added: polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4b.ll
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4b.ll?rev=302636&view=auto
> ==============================================================================
> --- polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4b.ll (added)
> +++ polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4b.ll Wed May 10 05:59:58 2017
> @@ -0,0 +1,55 @@
> +; RUN: opt %loadPolly -polly-scops -analyze < %s \
> +; RUN: -polly-invariant-load-hoisting \
> +; RUN: | FileCheck %s
> +
> +; Verify that two arrays delinearized with different sizes are not coalesced.
> +
> +; CHECK: Stmt_body1
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: [m, n] -> { Stmt_body1[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: [m, n] -> { Stmt_body1[i0] -> [i0, 0] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: [m, n] -> { Stmt_body1[i0] -> MemRef_baseB[i0, i0] };
> +; CHECK-NEXT: Stmt_body2
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: [m, n] -> { Stmt_body2[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: [m, n] -> { Stmt_body2[i0] -> [i0, 1] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: [m, n] -> { Stmt_body2[i0] -> MemRef_baseA[i0, i0] };
> +; CHECK-NEXT: }
> +
> +define void @foo(float** %A, i64 %n, i64 %m) {
> +start:
> + br label %loop
> +
> +loop:
> + %indvar = phi i64 [0, %start], [%indvar.next, %latch]
> + %indvar.next = add nsw i64 %indvar, 1
> + %icmp = icmp slt i64 %indvar.next, 1024
> + br i1 %icmp, label %body1, label %exit
> +
> +body1:
> + %baseB = load float*, float** %A
> + %offsetB = mul i64 %indvar, %m
> + %offsetB2 = add i64 %offsetB, %indvar
> + %ptrB = getelementptr float, float* %baseB, i64 %offsetB2
> + store float 42.0, float* %ptrB
> + br label %body2
> +
> +body2:
> + %baseA = load float*, float** %A
> + %offsetA = mul i64 %indvar, %n
> + %offsetA2 = add i64 %offsetA, %indvar
> + %ptrA = getelementptr float, float* %baseA, i64 %offsetA2
> + store float 42.0, float* %ptrA
> + br label %latch
> +
> +latch:
> + br label %loop
> +
> +exit:
> + ret void
> +
> +}
>
> Added: polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4c.ll
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4c.ll?rev=302636&view=auto
> ==============================================================================
> --- polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4c.ll (added)
> +++ polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_4c.ll Wed May 10 05:59:58 2017
> @@ -0,0 +1,51 @@
> +; RUN: opt %loadPolly -polly-scops -analyze < %s \
> +; RUN: -polly-invariant-load-hoisting \
> +; RUN: | FileCheck %s
> +
> +; Verify that arrays with different element types are not coalesced.
> +
> +; CHECK: Statements {
> +; CHECK-NEXT: Stmt_body1
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body1[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body1[i0] -> [i0, 0] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body1[i0] -> MemRef_baseB[0] };
> +; CHECK-NEXT: Stmt_body2
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body2[i0] : 0 <= i0 <= 1022 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body2[i0] -> [i0, 1] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body2[i0] -> MemRef_baseA[0] };
> +; CHECK-NEXT: }
> +
> +define void @foo(float** %A, i64 %n, i64 %m) {
> +start:
> + br label %loop
> +
> +loop:
> + %indvar = phi i64 [0, %start], [%indvar.next, %latch]
> + %indvar.next = add nsw i64 %indvar, 1
> + %icmp = icmp slt i64 %indvar.next, 1024
> + br i1 %icmp, label %body1, label %exit
> +
> +body1:
> + %baseB = load float*, float** %A
> + store float 42.0, float* %baseB
> + br label %body2
> +
> +body2:
> + %baseA = load float*, float** %A
> + %ptrcast = bitcast float* %baseA to i64*
> + store i64 42, i64* %ptrcast
> + br label %latch
> +
> +latch:
> + br label %loop
> +
> +exit:
> + ret void
> +
> +}
>
> Added: polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_5.ll
> URL: http://llvm.org/viewvc/llvm-project/polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_5.ll?rev=302636&view=auto
> ==============================================================================
> --- polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_5.ll (added)
> +++ polly/trunk/test/ScopInfo/invariant_load_canonicalize_array_baseptrs_5.ll Wed May 10 05:59:58 2017
> @@ -0,0 +1,78 @@
> +; RUN: opt %loadPolly -polly-scops -analyze < %s \
> +; RUN: -polly-invariant-load-hoisting \
> +; RUN: | FileCheck %s
> +
> +; Verify that nested arrays with invariant base pointers are handled correctly.
> +; Specifically, we currently do not canonicalize arrays where some accesses are
> +; hoisted as invariant loads. If we would, we need to update the access function
> +; of the invariant loads as well. However, as this is not a very common
> +; situation, we leave this for now to avoid further complexity increases.
> +;
> +; In this test case the arrays baseA1 and baseA2 could be canonicalized to a
> +; single array, but there is also an invariant access to baseA1[0] through
> +; "%v0 = load float, float* %ptr" which prevents the canonicalization.
> +
> +; CHECK: Invariant Accesses: {
> +; CHECK-NEXT: ReadAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body2[i0] -> MemRef_A[0] };
> +; CHECK-NEXT: Execution Context: { : }
> +; CHECK-NEXT: ReadAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body1[i0] -> MemRef_baseA1[0] };
> +; CHECK-NEXT: Execution Context: { : }
> +; CHECK-NEXT: }
> +
> +; CHECK: Statements {
> +; CHECK-NEXT: Stmt_body1
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body1[i0] : 0 <= i0 <= 1021 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body1[i0] -> [i0, 0] };
> +; CHECK-NEXT: ReadAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body1[i0] -> MemRef_baseA1[1 + i0] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body1[i0] -> MemRef_B[0] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body1[i0] -> MemRef_B[0] };
> +; CHECK-NEXT: Stmt_body2
> +; CHECK-NEXT: Domain :=
> +; CHECK-NEXT: { Stmt_body2[i0] : 0 <= i0 <= 1021 };
> +; CHECK-NEXT: Schedule :=
> +; CHECK-NEXT: { Stmt_body2[i0] -> [i0, 1] };
> +; CHECK-NEXT: MustWriteAccess := [Reduction Type: NONE] [Scalar: 0]
> +; CHECK-NEXT: { Stmt_body2[i0] -> MemRef_baseA2[0] };
> +; CHECK-NEXT: }
> +
> +define void @foo(float** %A, float* %B) {
> +start:
> + br label %loop
> +
> +loop:
> + %indvar = phi i64 [1, %start], [%indvar.next, %latch]
> + %indvar.next = add nsw i64 %indvar, 1
> + %icmp = icmp slt i64 %indvar.next, 1024
> + br i1 %icmp, label %body1, label %exit
> +
> +body1:
> + %baseA1 = load float*, float** %A
> + %ptr = getelementptr inbounds float, float* %baseA1, i64 %indvar
> + %v0 = load float, float* %ptr
> + %v1 = load float, float* %baseA1
> + store float %v0, float* %B
> + store float %v1, float* %B
> + br label %body2
> +
> +body2:
> + %baseA2 = load float*, float** %A
> + store float undef, float* %baseA2
> + br label %body3
> +
> +body3:
> + br label %latch
> +
> +latch:
> + br label %loop
> +
> +exit:
> + ret void
> +
> +}
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list