[polly] [Polly] Keep proximity dependence exact if simplifying it unbounds its distance (PR #225840)
Timur Baidusenov via llvm-commits
llvm-commits at lists.llvm.org
Sat Sep 26 10:32:41 PDT 2026
https://github.com/bai-tim updated https://github.com/llvm/llvm-project/pull/225840
>From 18e2e1690cc91b6f9c5ac4d472b8c9bc5718c684 Mon Sep 17 00:00:00 2001
From: Timur Baidusenov <timurbaidusenov at gmail.com>
Date: Wed, 23 Sep 2026 19:00:40 +0300
Subject: [PATCH 1/3] [Polly] Keep a proximity dependence exact if simplifying
it unbounds its distance
Before computing a schedule, the schedule optimizer removes the constraints implied by the iteration domains from the dependences. For a statement that reads a value written in one iteration of a loop in all later iterations, as floyd-warshall reads path[i][k] and path[k][j], this also removes the upper bound of the loop, so the proximity dependence has an unbounded distance along every schedule row that advances the loop; isl then finds no row whose distance it can bound, falls back to carrying the dependences one row at a time, and produces a wavefront over i + j instead of a permutable band of i and j, which is not tiled and traverses the array diagonally. This patch keeps the exact proximity dependence of a statement on itself if the simplified dependence has unbounded distances and the exact one has bounded distances, while the validity dependences and the dependences between different statements remain simplified. On PolyBench/C 4.2.1 LARGE (x86-64, -O3 -march=native -polly) floyd-warshall becomes 3.9 times faster (26.7 s to 6.8 s) and no other kernel changes; in llvm-test-suite only floyd-warshall (61 s to 16 s) and two small regression tests change, all tests pass, CTMark generates identical code, and the compile time of floyd-warshall grows from 131 to 134 ms.
Assisted-by: Claude (Anthropic)
---
polly/lib/Transform/ScheduleOptimizer.cpp | 37 ++++++++
.../simplify_deps_bounded_proximity.ll | 84 +++++++++++++++++++
2 files changed, 121 insertions(+)
create mode 100644 polly/test/ScheduleOptimizer/simplify_deps_bounded_proximity.ll
diff --git a/polly/lib/Transform/ScheduleOptimizer.cpp b/polly/lib/Transform/ScheduleOptimizer.cpp
index b9b9abbd85ae4..6db0d57f44841 100644
--- a/polly/lib/Transform/ScheduleOptimizer.cpp
+++ b/polly/lib/Transform/ScheduleOptimizer.cpp
@@ -656,6 +656,41 @@ static void printSchedule(llvm::raw_ostream &OS, const isl::schedule &Schedule,
}
#endif
+/// Return whether the dependence distances of @p Map, which relates instances
+/// of the same statement, are bounded.
+static bool hasBoundedDistances(const isl::map &Map) {
+ isl::set Deltas = Map.deltas();
+ return !Deltas.is_null() && isl_set_is_bounded(Deltas.get()) == isl_bool_true;
+}
+
+/// Undo the simplification of the proximity dependences of a statement on
+/// itself where it made their distances unbounded.
+///
+/// The scheduler looks for schedule rows that bound the distance of every
+/// proximity dependence. If the simplification drops the constraints of the
+/// domain that bound the distance of a dependence, such as a value that is
+/// read by all later iterations of a loop, then every row that advances along
+/// that loop has an unbounded distance, and the scheduler falls back to
+/// carrying dependences one row at a time instead of forming a permutable
+/// band.
+///
+/// @param Simplified The simplified proximity dependences.
+/// @param Exact The proximity dependences before simplification.
+static isl::union_map keepBoundedDistances(isl::union_map Simplified,
+ const isl::union_map &Exact) {
+ isl::union_map Result = isl::union_map::empty(Simplified.ctx());
+ for (isl::map Map : Simplified.get_map_list()) {
+ isl::space Space = Map.get_space();
+ if (Space.domain().is_equal(Space.range()) && !hasBoundedDistances(Map)) {
+ isl::map ExactMap = Exact.extract_map(Space);
+ if (hasBoundedDistances(ExactMap))
+ Map = ExactMap;
+ }
+ Result = Result.unite(isl::union_map(Map));
+ }
+ return Result;
+}
+
/// Collect statistics for the schedule tree.
///
/// @param Schedule The schedule tree to analyze. If not a schedule tree it is
@@ -814,10 +849,12 @@ static void runIslScheduleOptimizerImpl(
// interesting anyway. In some cases this option may stop the scheduler to
// find any schedule.
if (SimplifyDeps == "yes") {
+ isl::union_map ExactProximity = Proximity;
Validity = Validity.gist_domain(Domain);
Validity = Validity.gist_range(Domain);
Proximity = Proximity.gist_domain(Domain);
Proximity = Proximity.gist_range(Domain);
+ Proximity = keepBoundedDistances(Proximity, ExactProximity);
} else if (SimplifyDeps != "no") {
errs()
<< "warning: Option -polly-opt-simplify-deps should either be 'yes' "
diff --git a/polly/test/ScheduleOptimizer/simplify_deps_bounded_proximity.ll b/polly/test/ScheduleOptimizer/simplify_deps_bounded_proximity.ll
new file mode 100644
index 0000000000000..ccf2a4e405a74
--- /dev/null
+++ b/polly/test/ScheduleOptimizer/simplify_deps_bounded_proximity.ll
@@ -0,0 +1,84 @@
+; RUN: opt %loadNPMPolly '-passes=polly-custom<opt-isl>' -polly-print-opt-isl -disable-output < %s | FileCheck %s
+;
+; Keep the exact proximity dependence of a statement on itself if simplifying
+; it makes its distances unbounded.
+;
+; void floyd(int n, int path[restrict n][n]) {
+; for (int k = 0; k < n; k++)
+; for (int i = 0; i < n; i++)
+; for (int j = 0; j < n; j++)
+; path[i][j] = path[i][j] < path[i][k] + path[k][j]
+; ? path[i][j]
+; : path[i][k] + path[k][j];
+; }
+;
+; Within an iteration of k, path[i][k] is read by all later iterations of j
+; and path[k][j] by all later iterations of i. The simplified dependences lose
+; the upper bound of j and i, so no schedule row along i or j bounds their
+; distance. The scheduler then carried the dependences with a wavefront over
+; i + j instead of forming a permutable band of i and j.
+
+; CHECK: schedule: "[n] -> [{ Stmt_for_body8[i0, i1, i2] -> [(i0)] }]"
+; CHECK-NEXT: permutable: 1
+; CHECK-NEXT: child:
+; CHECK-NEXT: mark: "1st level tiling - Tiles"
+; CHECK-NEXT: child:
+; CHECK-NEXT: schedule: "[n] -> [{ Stmt_for_body8[i0, i1, i2] -> [(floor((i1)/32))] }, { Stmt_for_body8[i0, i1, i2] -> [(floor((i2)/32))] }]"
+; CHECK-NEXT: permutable: 1
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+
+define void @floyd(i32 %n, ptr noalias %path) {
+entry:
+ %0 = zext i32 %n to i64
+ %cmp74 = icmp sgt i32 %n, 0
+ br i1 %cmp74, label %for.cond1.preheader.preheader, label %for.cond.cleanup
+
+for.cond1.preheader.preheader:
+ %wide.trip.count85 = zext nneg i32 %n to i64
+ %wide.trip.count80 = zext nneg i32 %n to i64
+ %wide.trip.count = zext nneg i32 %n to i64
+ br label %for.cond1.preheader
+
+for.cond1.preheader:
+ %indvars.iv82 = phi i64 [ 0, %for.cond1.preheader.preheader ], [ %indvars.iv.next83, %for.cond1.for.cond.cleanup3_crit_edge ]
+ %1 = mul nuw nsw i64 %indvars.iv82, %0
+ %arrayidx16 = getelementptr inbounds nuw [4 x i8], ptr %path, i64 %1
+ br label %for.cond5.preheader
+
+for.cond.cleanup:
+ ret void
+
+for.cond5.preheader:
+ %indvars.iv77 = phi i64 [ 0, %for.cond1.preheader ], [ %indvars.iv.next78, %for.cond5.for.cond.cleanup7_crit_edge ]
+ %2 = mul nuw nsw i64 %indvars.iv77, %0
+ %arrayidx = getelementptr inbounds nuw [4 x i8], ptr %path, i64 %2
+ %arrayidx14 = getelementptr inbounds nuw [4 x i8], ptr %arrayidx, i64 %indvars.iv82
+ br label %for.body8
+
+for.cond1.for.cond.cleanup3_crit_edge:
+ %indvars.iv.next83 = add nuw nsw i64 %indvars.iv82, 1
+ %exitcond86.not = icmp eq i64 %indvars.iv.next83, %wide.trip.count85
+ br i1 %exitcond86.not, label %for.cond.cleanup, label %for.cond1.preheader
+
+for.cond5.for.cond.cleanup7_crit_edge:
+ %indvars.iv.next78 = add nuw nsw i64 %indvars.iv77, 1
+ %exitcond81.not = icmp eq i64 %indvars.iv.next78, %wide.trip.count80
+ br i1 %exitcond81.not, label %for.cond1.for.cond.cleanup3_crit_edge, label %for.cond5.preheader
+
+for.body8:
+ %indvars.iv = phi i64 [ 0, %for.cond5.preheader ], [ %indvars.iv.next, %for.body8 ]
+ %arrayidx10 = getelementptr inbounds nuw [4 x i8], ptr %arrayidx, i64 %indvars.iv
+ %3 = load i32, ptr %arrayidx10, align 4
+ %4 = load i32, ptr %arrayidx14, align 4
+ %arrayidx18 = getelementptr inbounds nuw [4 x i8], ptr %arrayidx16, i64 %indvars.iv
+ %5 = load i32, ptr %arrayidx18, align 4
+ %add = add nsw i32 %5, %4
+ %.add = tail call i32 @llvm.smin.i32(i32 %3, i32 %add)
+ store i32 %.add, ptr %arrayidx10, align 4
+ %indvars.iv.next = add nuw nsw i64 %indvars.iv, 1
+ %exitcond.not = icmp eq i64 %indvars.iv.next, %wide.trip.count
+ br i1 %exitcond.not, label %for.cond5.for.cond.cleanup7_crit_edge, label %for.body8
+}
+
+declare i32 @llvm.smin.i32(i32, i32)
>From f27840008a7f54141b4118f24afb5b582d654d0a Mon Sep 17 00:00:00 2001
From: Timur Baidusenov <timurbaidusenov at gmail.com>
Date: Thu, 24 Sep 2026 18:46:10 +0300
Subject: [PATCH 2/3] Address review
Intersect the simplified dependence with the translation by the hull of the exact distances instead of replacing it by the exact dependence, and fall back to the exact dependence only if this does not bound the distances. Use isl::set::is_bounded and pass the simplified dependences by reference.
Assisted-by: Claude (Anthropic)
---
polly/lib/Transform/ScheduleOptimizer.cpp | 13 ++++++++++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/polly/lib/Transform/ScheduleOptimizer.cpp b/polly/lib/Transform/ScheduleOptimizer.cpp
index 6db0d57f44841..07d16ffbdb597 100644
--- a/polly/lib/Transform/ScheduleOptimizer.cpp
+++ b/polly/lib/Transform/ScheduleOptimizer.cpp
@@ -660,7 +660,7 @@ static void printSchedule(llvm::raw_ostream &OS, const isl::schedule &Schedule,
/// of the same statement, are bounded.
static bool hasBoundedDistances(const isl::map &Map) {
isl::set Deltas = Map.deltas();
- return !Deltas.is_null() && isl_set_is_bounded(Deltas.get()) == isl_bool_true;
+ return !Deltas.is_null() && Deltas.is_bounded().is_true();
}
/// Undo the simplification of the proximity dependences of a statement on
@@ -676,14 +676,21 @@ static bool hasBoundedDistances(const isl::map &Map) {
///
/// @param Simplified The simplified proximity dependences.
/// @param Exact The proximity dependences before simplification.
-static isl::union_map keepBoundedDistances(isl::union_map Simplified,
+static isl::union_map keepBoundedDistances(const isl::union_map &Simplified,
const isl::union_map &Exact) {
isl::union_map Result = isl::union_map::empty(Simplified.ctx());
for (isl::map Map : Simplified.get_map_list()) {
isl::space Space = Map.get_space();
if (Space.domain().is_equal(Space.range()) && !hasBoundedDistances(Map)) {
+ // Only add the constraints that bound the distances before the
+ // simplification, i.e. the hull of the exact distances, rather than
+ // restoring all constraints of the exact dependence.
isl::map ExactMap = Exact.extract_map(Space);
- if (hasBoundedDistances(ExactMap))
+ isl::map Bounded =
+ Map.intersect(ExactMap.deltas().simple_hull().translation());
+ if (hasBoundedDistances(Bounded))
+ Map = Bounded;
+ else if (hasBoundedDistances(ExactMap))
Map = ExactMap;
}
Result = Result.unite(isl::union_map(Map));
>From 4f70283401e2ae69543ed6f6804ba72bd1d9dc33 Mon Sep 17 00:00:00 2001
From: Timur Baidusenov <timurbaidusenov at gmail.com>
Date: Sat, 26 Sep 2026 20:26:35 +0300
Subject: [PATCH 3/3] Address review
If the hull of the exact distances does not bound the distances of the simplified dependence, intersect it with the translation by the exact distances instead of replacing it by the exact dependence, so that the domain constraints removed by the simplification are not restored in this case either.
Assisted-by: Claude (Anthropic)
---
polly/lib/Transform/ScheduleOptimizer.cpp | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/polly/lib/Transform/ScheduleOptimizer.cpp b/polly/lib/Transform/ScheduleOptimizer.cpp
index 07d16ffbdb597..d6dac4a0ee2f3 100644
--- a/polly/lib/Transform/ScheduleOptimizer.cpp
+++ b/polly/lib/Transform/ScheduleOptimizer.cpp
@@ -683,15 +683,15 @@ static isl::union_map keepBoundedDistances(const isl::union_map &Simplified,
isl::space Space = Map.get_space();
if (Space.domain().is_equal(Space.range()) && !hasBoundedDistances(Map)) {
// Only add the constraints that bound the distances before the
- // simplification, i.e. the hull of the exact distances, rather than
- // restoring all constraints of the exact dependence.
- isl::map ExactMap = Exact.extract_map(Space);
- isl::map Bounded =
- Map.intersect(ExactMap.deltas().simple_hull().translation());
+ // simplification rather than restoring all constraints of the exact
+ // dependence: preferably the hull of the exact distances, which is a
+ // single convex set, otherwise the exact distances themselves.
+ isl::set ExactDeltas = Exact.extract_map(Space).deltas();
+ isl::map Bounded = Map.intersect(ExactDeltas.simple_hull().translation());
+ if (!hasBoundedDistances(Bounded))
+ Bounded = Map.intersect(ExactDeltas.translation());
if (hasBoundedDistances(Bounded))
Map = Bounded;
- else if (hasBoundedDistances(ExactMap))
- Map = ExactMap;
}
Result = Result.unite(isl::union_map(Map));
}
More information about the llvm-commits
mailing list