[flang-commits] [flang] [llvm] [Pipeline] Add IPSCCPPass at O3 to fix function specialization phase ordering (PR #201543)
via flang-commits
flang-commits at lists.llvm.org
Fri Aug 21 12:02:04 PDT 2026
https://github.com/anoopkg6 updated https://github.com/llvm/llvm-project/pull/201543
>From ecbdc6dcce08c329676766bf2a6452c94319c02a Mon Sep 17 00:00:00 2001
From: "anoop.kumar6 at ibm.com" <anoopk at b35lp63.lnxne.boe>
Date: Thu, 4 Jun 2026 00:17:25 +0200
Subject: [PATCH 1/4] [Pipeline] Add IPSCCPPass at O3 to fix function
specialization Phase ordering
Add a second IPSCCPPass with function specialization enabled at -O3
directly after the inliner pipeline during non-LTO phases. This
addresses a target-independent phase ordering issue stemming from
Fortran's pass-by-reference semantics.
Introducing IPSCCPPass after buildInlinerPipeline allows LoopRotate to
rewrite the control flow topology from a while-loop to a do-while
layout. This structural inversion duplicates the first execution of
the loop body and pulls it outside the cyclic boundaries into a
single-predecessor entry block, splitting the critical edge. Memory
disambiguation then resolves in straight-line code, allowing the late
IPSCCP run to successfully evaluate constant initializations that were
previously blocked.
Cross-compilation check using --target=x86_64-unknown-linux-gnu
confirms that this phase ordering issue is target-independent. Function
specialization fails to trigger without this pipeline modification.
Performance Impact on SystemZ (548.exchange2_r):
- Patched (Before addVectorPasses): ~56% performance improvement.
- Delayed Placement (After addVectorPasses): Only ~16% improvement.
Compile-Time & Run-Time Impact on SystemZ (SPEC CPU 2017):
- Total Suite Runtime: ~1.75% geometric mean improvement.
- Total Suite Compile-Time: ~0% impact (-0.09% geometric mean delta).
Placing the pass early delivers the full optimization gain, whereas
placing it after vectorization drops the gain significantly, proving
that earlier execution is critical to avoid degrading downstream
vectorization.
---
...spec-phase-ordering-loop-rotate-ipsccp.f90 | 50 +++++++++++++++++++
llvm/lib/Passes/PassBuilderPipelines.cpp | 6 +++
llvm/test/Other/new-pm-defaults.ll | 5 +-
...ction-specialization-loop-rotate-ipsccp.ll | 48 ++++++++++++++++++
.../dce-after-argument-promotion.ll | 7 ++-
5 files changed, 111 insertions(+), 5 deletions(-)
create mode 100644 flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90
create mode 100644 llvm/test/Transforms/FunctionSpecialization/function-specialization-loop-rotate-ipsccp.ll
diff --git a/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90 b/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90
new file mode 100644
index 0000000000000..b18553b3bfd4b
--- /dev/null
+++ b/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90
@@ -0,0 +1,50 @@
+! RUN: %flang -O2 -mllvm -force-specialization -Xflang -fdebug-pass-manager -S -emit-llvm %s -o %t.O2.ll 2>%t.O2.stderr
+! RUN: FileCheck %s --check-prefix=PASS-O2-PIPE < %t.O2.stderr
+! RUN: FileCheck %s --check-prefix=PASS-O2-IR < %t.O2.ll
+!
+! RUN: %flang -O3 -mllvm -force-specialization -Xflang -fdebug-pass-manager -S -emit-llvm %s -o %t.O3.ll 2>%t.O3.stderr
+! RUN: FileCheck %s --check-prefix=PASS-O3-PIPE < %t.O3.stderr
+! RUN: FileCheck %s --check-prefix=PASS-O3-IR < %t.O3.ll
+
+module brute_force
+ implicit none
+ integer, public :: fallback_sink
+ integer, volatile, public :: global_fence = 1
+contains
+ subroutine top_level_caller()
+ integer :: temp
+ temp = 2
+ call digits_2(temp)
+ end subroutine top_level_caller
+
+ recursive subroutine digits_2(arg1)
+ integer, intent(in) :: arg1
+ integer :: temp_inner
+
+ if (global_fence == 1) then
+ if (arg1 == 2) then
+ temp_inner = arg1
+ call digits_2(temp_inner)
+ else
+ fallback_sink = arg1 * 5 + 12
+ end if
+ end if
+ end subroutine digits_2
+end module brute_force
+
+! PASS-O2-PIPE: Running pass: IPSCCPPass on [module]
+! PASS-O2-PIPE: Running pass: InlinerPass on (
+! PASS-O2-PIPE-NOT: Running pass: IPSCCPPass on [module]
+! PASS-O2-PIPE: Running pass: DeadArgumentEliminationPass on [module]
+
+! PASS-O2-IR-NOT: .specialized.
+
+! PASS-O3-PIPE: Running pass: IPSCCPPass on [module]
+! PASS-O3-PIPE: Running pass: InlinerPass on (
+! PASS-O3-PIPE: Running pass: IPSCCPPass on [module]
+! PASS-O3-PIPE: Running pass: DeadArgumentEliminationPass on [module]
+
+! PASS-O3-IR: define void @{{.*}}top_level_caller
+! PASS-O3-IR: call fastcc void @{{.*}}digits_2{{.*}}.specialized.{{.*}}()
+! PASS-O3-IR: define internal fastcc void @{{.*}}digits_2{{.*}}.specialized.1()
+! PASS-O3-IR: define internal fastcc void @{{.*}}digits_2{{.*}}.specialized.2()
diff --git a/llvm/lib/Passes/PassBuilderPipelines.cpp b/llvm/lib/Passes/PassBuilderPipelines.cpp
index b8c5b1eab2f97..8d272ccb2487a 100644
--- a/llvm/lib/Passes/PassBuilderPipelines.cpp
+++ b/llvm/lib/Passes/PassBuilderPipelines.cpp
@@ -1322,6 +1322,12 @@ PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
else
MPM.addPass(buildInlinerPipeline(Level, Phase));
+ // Run a second IPSCCP pass at O3 to resolve function specialization
+ // opportunities exposed by loop rotation reference parameters
+ // (e.g., Fortran pass-by-reference).
+ if (Level.getSpeedupLevel() >= 3 && Phase == ThinOrFullLTOPhase::None)
+ MPM.addPass(llvm::IPSCCPPass(IPSCCPOptions(/*AllowFuncSpec=*/true)));
+
// Remove any dead arguments exposed by cleanups, constant folding globals,
// and argument promotion.
MPM.addPass(DeadArgumentEliminationPass());
diff --git a/llvm/test/Other/new-pm-defaults.ll b/llvm/test/Other/new-pm-defaults.ll
index 2c1a57e00c9f1..c35b293f46cfd 100644
--- a/llvm/test/Other/new-pm-defaults.ll
+++ b/llvm/test/Other/new-pm-defaults.ll
@@ -218,7 +218,10 @@
; CHECK-O-NEXT: Running pass: InvalidateAnalysisPass<{{.*}}ShouldNotRunFunctionPassesAnalysis
; CHECK-O-NEXT: Invalidating analysis: ShouldNotRunFunctionPassesAnalysis
; CHECK-O-NEXT: Invalidating analysis: InlineAdvisorAnalysis
-; CHECK-O-NEXT: Running pass: DeadArgumentEliminationPass
+; CHECK-O1-NEXT: Running pass: DeadArgumentEliminationPass
+; CHECK-O2-NEXT: Running pass: DeadArgumentEliminationPass
+; CHECK-O3-NOEXT: Running pass: IPSCCPPass
+; CHECK-O3: Running pass: DeadArgumentEliminationPass
; CHECK-O-NEXT: Running pass: CoroCleanupPass
; CHECK-O-NEXT: Running pass: GlobalOptPass
; CHECK-O-NEXT: Running pass: GlobalDCEPass
diff --git a/llvm/test/Transforms/FunctionSpecialization/function-specialization-loop-rotate-ipsccp.ll b/llvm/test/Transforms/FunctionSpecialization/function-specialization-loop-rotate-ipsccp.ll
new file mode 100644
index 0000000000000..d80c69344b8c0
--- /dev/null
+++ b/llvm/test/Transforms/FunctionSpecialization/function-specialization-loop-rotate-ipsccp.ll
@@ -0,0 +1,48 @@
+; RUN: opt -S -passes='ipsccp,function(loop(loop-rotate))' -force-specialization < %s | FileCheck %s --check-prefix=NO-LATE-IPSCCP
+; RUN: opt -S -passes='ipsccp,function(loop(loop-rotate)),ipsccp' -force-specialization < %s | FileCheck %s --check-prefix=WITH-LATE-IPSCCP
+
+ at external_cond = external global i1, align 1
+
+define void @top_level_caller() {
+; WITH-LATE-IPSCCP-LABEL: define void @top_level_caller(
+; WITH-LATE-IPSCCP: call void @digits_2.specialized.{{[0-9]+}}(
+
+; NO-LATE-IPSCCP-LABEL: define void @top_level_caller(
+; NO-LATE-IPSCCP-NOT: @digits_2.specialized
+entry:
+ %temp = alloca i32, align 4
+ store i32 2, ptr %temp, align 4
+ %cond = load i1, ptr @external_cond, align 1
+ %idx = select i1 %cond, i32 0, i32 99
+ call void @digits_2(ptr %temp, i32 %idx)
+ ret void
+}
+
+define internal void @digits_2(ptr %arg1, i32 %loop_idx) {
+; NO-LATE-IPSCCP-LABEL: define internal void @digits_2(
+; NO-LATE-IPSCCP: %val1 = load i32, ptr %arg1
+; NO-LATE-IPSCCP: br label %loop.body
+
+; WITH-LATE-IPSCCP-LABEL: define internal void @digits_2.specialized.{{[0-9]+}}(
+entry:
+ %temp_alloc = alloca i32, align 4
+ br label %loop.header
+
+loop.header:
+ %ptr_input = phi ptr [ %arg1, %entry ], [ %arrayidx, %loop.body ]
+ %iv = phi i32 [ %loop_idx, %entry ], [ %iv.next, %loop.body ]
+ %val = load i32, ptr %ptr_input, align 4
+ %cmp = icmp slt i32 %iv, 1
+ br i1 %cmp, label %loop.body, label %exit
+
+loop.body:
+ %idxprom = sext i32 %iv to i64
+ %arrayidx = getelementptr inbounds i32, ptr %ptr_input, i64 %idxprom
+ store i32 %val, ptr %arrayidx, align 4
+ %iv.next = add nsw i32 %iv, 1
+ call void @digits_2(ptr %temp_alloc, i32 %iv.next)
+ br label %loop.header
+
+exit:
+ ret void
+}
diff --git a/llvm/test/Transforms/PhaseOrdering/dce-after-argument-promotion.ll b/llvm/test/Transforms/PhaseOrdering/dce-after-argument-promotion.ll
index c33fcfbe6ed97..435112fccb744 100644
--- a/llvm/test/Transforms/PhaseOrdering/dce-after-argument-promotion.ll
+++ b/llvm/test/Transforms/PhaseOrdering/dce-after-argument-promotion.ll
@@ -9,10 +9,9 @@
define internal void @f(ptr byval(%struct.ss) align 8 %b, ptr byval(i32) align 4 %X) noinline nounwind {
; CHECK-LABEL: define {{[^@]+}}@f
-; CHECK-SAME: (i32 [[B_0:%.*]]){{[^#]*}} #[[ATTR0:[0-9]+]] {
+; CHECK-SAME: (){{[^#]*}} #[[ATTR0:[0-9]+]] {
; CHECK-NEXT: entry:
-; CHECK-NEXT: [[TEMP:%.*]] = add i32 [[B_0]], 1
-; CHECK-NEXT: store i32 [[TEMP]], ptr [[DUMMY]], align 4
+; CHECK-NEXT: store i32 2, ptr @dummy, align 4
; CHECK-NEXT: ret void
;
entry:
@@ -27,7 +26,7 @@ define i32 @test(ptr %X) {
; CHECK-LABEL: define {{[^@]+}}@test
; CHECK-SAME: (ptr {{[^%]*}} [[X:%.*]]){{[^#]*}} #[[ATTR1:[0-9]+]] {
; CHECK-NEXT: entry:
-; CHECK-NEXT: tail call {{.*}}void @f(i32 1)
+; CHECK-NEXT: tail call fastcc void @f()
; CHECK-NEXT: ret i32 0
;
entry:
>From 39a882ff8ca666ab2a3bf8e4ea9d488baa011a78 Mon Sep 17 00:00:00 2001
From: "anoop.kumar6 at ibm.com" <Anoop.Kumar6 at ibm.com>
Date: Thu, 30 Jul 2026 15:58:55 +0200
Subject: [PATCH 2/4] [Pipeline] Move loop-rotate phase ordering test to
PhaseOrdering/
The test depends on loop-rotate and ipsccp interaction, so PhaseOrdering/
is the correct location. Update FileCheck patterns to cover the full IR.
---
...spec-phase-ordering-loop-rotate-ipsccp.f90 | 6 +-
llvm/lib/Passes/PassBuilderPipelines.cpp | 2 +-
...ction-specialization-loop-rotate-ipsccp.ll | 48 ---------
.../ipsccp-loop-rotate-funcspec.ll | 100 ++++++++++++++++++
4 files changed, 103 insertions(+), 53 deletions(-)
delete mode 100644 llvm/test/Transforms/FunctionSpecialization/function-specialization-loop-rotate-ipsccp.ll
create mode 100644 llvm/test/Transforms/PhaseOrdering/ipsccp-loop-rotate-funcspec.ll
diff --git a/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90 b/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90
index b18553b3bfd4b..045ca422ab55c 100644
--- a/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90
+++ b/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90
@@ -37,7 +37,7 @@ end module brute_force
! PASS-O2-PIPE-NOT: Running pass: IPSCCPPass on [module]
! PASS-O2-PIPE: Running pass: DeadArgumentEliminationPass on [module]
-! PASS-O2-IR-NOT: .specialized.
+! PASS-O2-IR-NOT: define {{.*}}@{{.*}}.specialized.
! PASS-O3-PIPE: Running pass: IPSCCPPass on [module]
! PASS-O3-PIPE: Running pass: InlinerPass on (
@@ -45,6 +45,4 @@ end module brute_force
! PASS-O3-PIPE: Running pass: DeadArgumentEliminationPass on [module]
! PASS-O3-IR: define void @{{.*}}top_level_caller
-! PASS-O3-IR: call fastcc void @{{.*}}digits_2{{.*}}.specialized.{{.*}}()
-! PASS-O3-IR: define internal fastcc void @{{.*}}digits_2{{.*}}.specialized.1()
-! PASS-O3-IR: define internal fastcc void @{{.*}}digits_2{{.*}}.specialized.2()
+! PASS-O3-IR: label %{{.*}}digits_2{{.*}}.specialized.
diff --git a/llvm/lib/Passes/PassBuilderPipelines.cpp b/llvm/lib/Passes/PassBuilderPipelines.cpp
index 8d272ccb2487a..59235c0a6a672 100644
--- a/llvm/lib/Passes/PassBuilderPipelines.cpp
+++ b/llvm/lib/Passes/PassBuilderPipelines.cpp
@@ -1325,7 +1325,7 @@ PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
// Run a second IPSCCP pass at O3 to resolve function specialization
// opportunities exposed by loop rotation reference parameters
// (e.g., Fortran pass-by-reference).
- if (Level.getSpeedupLevel() >= 3 && Phase == ThinOrFullLTOPhase::None)
+ if (Level == OptimizationLevel::O3 && Phase == ThinOrFullLTOPhase::None)
MPM.addPass(llvm::IPSCCPPass(IPSCCPOptions(/*AllowFuncSpec=*/true)));
// Remove any dead arguments exposed by cleanups, constant folding globals,
diff --git a/llvm/test/Transforms/FunctionSpecialization/function-specialization-loop-rotate-ipsccp.ll b/llvm/test/Transforms/FunctionSpecialization/function-specialization-loop-rotate-ipsccp.ll
deleted file mode 100644
index d80c69344b8c0..0000000000000
--- a/llvm/test/Transforms/FunctionSpecialization/function-specialization-loop-rotate-ipsccp.ll
+++ /dev/null
@@ -1,48 +0,0 @@
-; RUN: opt -S -passes='ipsccp,function(loop(loop-rotate))' -force-specialization < %s | FileCheck %s --check-prefix=NO-LATE-IPSCCP
-; RUN: opt -S -passes='ipsccp,function(loop(loop-rotate)),ipsccp' -force-specialization < %s | FileCheck %s --check-prefix=WITH-LATE-IPSCCP
-
- at external_cond = external global i1, align 1
-
-define void @top_level_caller() {
-; WITH-LATE-IPSCCP-LABEL: define void @top_level_caller(
-; WITH-LATE-IPSCCP: call void @digits_2.specialized.{{[0-9]+}}(
-
-; NO-LATE-IPSCCP-LABEL: define void @top_level_caller(
-; NO-LATE-IPSCCP-NOT: @digits_2.specialized
-entry:
- %temp = alloca i32, align 4
- store i32 2, ptr %temp, align 4
- %cond = load i1, ptr @external_cond, align 1
- %idx = select i1 %cond, i32 0, i32 99
- call void @digits_2(ptr %temp, i32 %idx)
- ret void
-}
-
-define internal void @digits_2(ptr %arg1, i32 %loop_idx) {
-; NO-LATE-IPSCCP-LABEL: define internal void @digits_2(
-; NO-LATE-IPSCCP: %val1 = load i32, ptr %arg1
-; NO-LATE-IPSCCP: br label %loop.body
-
-; WITH-LATE-IPSCCP-LABEL: define internal void @digits_2.specialized.{{[0-9]+}}(
-entry:
- %temp_alloc = alloca i32, align 4
- br label %loop.header
-
-loop.header:
- %ptr_input = phi ptr [ %arg1, %entry ], [ %arrayidx, %loop.body ]
- %iv = phi i32 [ %loop_idx, %entry ], [ %iv.next, %loop.body ]
- %val = load i32, ptr %ptr_input, align 4
- %cmp = icmp slt i32 %iv, 1
- br i1 %cmp, label %loop.body, label %exit
-
-loop.body:
- %idxprom = sext i32 %iv to i64
- %arrayidx = getelementptr inbounds i32, ptr %ptr_input, i64 %idxprom
- store i32 %val, ptr %arrayidx, align 4
- %iv.next = add nsw i32 %iv, 1
- call void @digits_2(ptr %temp_alloc, i32 %iv.next)
- br label %loop.header
-
-exit:
- ret void
-}
diff --git a/llvm/test/Transforms/PhaseOrdering/ipsccp-loop-rotate-funcspec.ll b/llvm/test/Transforms/PhaseOrdering/ipsccp-loop-rotate-funcspec.ll
new file mode 100644
index 0000000000000..9ad1d55f077f2
--- /dev/null
+++ b/llvm/test/Transforms/PhaseOrdering/ipsccp-loop-rotate-funcspec.ll
@@ -0,0 +1,100 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 6
+; RUN: opt -S -passes='ipsccp,function(loop(loop-rotate))' -force-specialization < %s | FileCheck %s --check-prefix=NO-LATE-IPSCCP
+; RUN: opt -S -passes='ipsccp,function(loop(loop-rotate)),ipsccp' -force-specialization < %s | FileCheck %s --check-prefix=WITH-LATE-IPSCCP
+
+ at external_cond = external global i1, align 1
+
+define void @top_level_caller() {
+; NO-LATE-IPSCCP-LABEL: define void @top_level_caller() {
+; NO-LATE-IPSCCP-NEXT: [[ENTRY:.*:]]
+; NO-LATE-IPSCCP-NEXT: [[TEMP:%.*]] = alloca i32, align 4
+; NO-LATE-IPSCCP-NEXT: store i32 2, ptr [[TEMP]], align 4
+; NO-LATE-IPSCCP-NEXT: [[COND:%.*]] = load i1, ptr @external_cond, align 1
+; NO-LATE-IPSCCP-NEXT: [[IDX:%.*]] = select i1 [[COND]], i32 0, i32 99
+; NO-LATE-IPSCCP-NEXT: call void @digits_2(ptr [[TEMP]], i32 [[IDX]])
+; NO-LATE-IPSCCP-NEXT: ret void
+;
+; WITH-LATE-IPSCCP-LABEL: define void @top_level_caller() {
+; WITH-LATE-IPSCCP-NEXT: [[ENTRY:.*:]]
+; WITH-LATE-IPSCCP-NEXT: [[TEMP:%.*]] = alloca i32, align 4
+; WITH-LATE-IPSCCP-NEXT: store i32 2, ptr [[TEMP]], align 4
+; WITH-LATE-IPSCCP-NEXT: [[COND:%.*]] = load i1, ptr @external_cond, align 1
+; WITH-LATE-IPSCCP-NEXT: [[IDX:%.*]] = select i1 [[COND]], i32 0, i32 99
+; WITH-LATE-IPSCCP-NEXT: call void @digits_2(ptr [[TEMP]], i32 [[IDX]])
+; WITH-LATE-IPSCCP-NEXT: ret void
+;
+entry:
+ %temp = alloca i32, align 4
+ store i32 2, ptr %temp, align 4
+ %cond = load i1, ptr @external_cond, align 1
+ %idx = select i1 %cond, i32 0, i32 99
+ call void @digits_2(ptr %temp, i32 %idx)
+ ret void
+}
+
+define internal void @digits_2(ptr %arg1, i32 %loop_idx) {
+; NO-LATE-IPSCCP-LABEL: define internal void @digits_2(
+; NO-LATE-IPSCCP-SAME: ptr nonnull [[ARG1:%.*]], i32 range(i32 0, 100) [[LOOP_IDX:%.*]]) {
+; NO-LATE-IPSCCP-NEXT: [[ENTRY:.*:]]
+; NO-LATE-IPSCCP-NEXT: [[TEMP_ALLOC:%.*]] = alloca i32, align 4
+; NO-LATE-IPSCCP-NEXT: [[VAL1:%.*]] = load i32, ptr [[ARG1]], align 4
+; NO-LATE-IPSCCP-NEXT: [[CMP2:%.*]] = icmp slt i32 [[LOOP_IDX]], 1
+; NO-LATE-IPSCCP-NEXT: br i1 [[CMP2]], label %[[LOOP_BODY_LR_PH:.*]], label %[[EXIT:.*]]
+; NO-LATE-IPSCCP: [[LOOP_BODY_LR_PH]]:
+; NO-LATE-IPSCCP-NEXT: br label %[[LOOP_BODY:.*]]
+; NO-LATE-IPSCCP: [[LOOP_BODY]]:
+; NO-LATE-IPSCCP-NEXT: [[VAL4:%.*]] = phi i32 [ [[VAL1]], %[[LOOP_BODY_LR_PH]] ], [ [[VAL:%.*]], %[[LOOP_BODY]] ]
+; NO-LATE-IPSCCP-NEXT: [[PTR_INPUT3:%.*]] = phi ptr [ [[ARG1]], %[[LOOP_BODY_LR_PH]] ], [ [[ARRAYIDX:%.*]], %[[LOOP_BODY]] ]
+; NO-LATE-IPSCCP-NEXT: [[ARRAYIDX]] = getelementptr inbounds nuw i32, ptr [[PTR_INPUT3]], i64 0
+; NO-LATE-IPSCCP-NEXT: store i32 [[VAL4]], ptr [[ARRAYIDX]], align 4
+; NO-LATE-IPSCCP-NEXT: call void @digits_2.specialized.1(ptr [[TEMP_ALLOC]], i32 1)
+; NO-LATE-IPSCCP-NEXT: [[VAL]] = load i32, ptr [[ARRAYIDX]], align 4
+; NO-LATE-IPSCCP-NEXT: [[CMP:%.*]] = icmp slt i32 1, 1
+; NO-LATE-IPSCCP-NEXT: br i1 [[CMP]], label %[[LOOP_BODY]], label %[[LOOP_HEADER_EXIT_CRIT_EDGE:.*]]
+; NO-LATE-IPSCCP: [[LOOP_HEADER_EXIT_CRIT_EDGE]]:
+; NO-LATE-IPSCCP-NEXT: br label %[[EXIT]]
+; NO-LATE-IPSCCP: [[EXIT]]:
+; NO-LATE-IPSCCP-NEXT: ret void
+;
+; WITH-LATE-IPSCCP-LABEL: define internal void @digits_2(
+; WITH-LATE-IPSCCP-SAME: ptr nonnull [[ARG1:%.*]], i32 range(i32 0, 100) [[LOOP_IDX:%.*]]) {
+; WITH-LATE-IPSCCP-NEXT: [[ENTRY:.*:]]
+; WITH-LATE-IPSCCP-NEXT: [[TEMP_ALLOC:%.*]] = alloca i32, align 4
+; WITH-LATE-IPSCCP-NEXT: [[VAL1:%.*]] = load i32, ptr [[ARG1]], align 4
+; WITH-LATE-IPSCCP-NEXT: [[CMP2:%.*]] = icmp slt i32 [[LOOP_IDX]], 1
+; WITH-LATE-IPSCCP-NEXT: br i1 [[CMP2]], label %[[LOOP_BODY_LR_PH:.*]], label %[[EXIT:.*]]
+; WITH-LATE-IPSCCP: [[LOOP_BODY_LR_PH]]:
+; WITH-LATE-IPSCCP-NEXT: br label %[[LOOP_BODY:.*]]
+; WITH-LATE-IPSCCP: [[LOOP_BODY]]:
+; WITH-LATE-IPSCCP-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds nuw i32, ptr [[ARG1]], i64 0
+; WITH-LATE-IPSCCP-NEXT: store i32 [[VAL1]], ptr [[ARRAYIDX]], align 4
+; WITH-LATE-IPSCCP-NEXT: call void @digits_2.specialized.1(ptr [[TEMP_ALLOC]], i32 1)
+; WITH-LATE-IPSCCP-NEXT: [[VAL:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
+; WITH-LATE-IPSCCP-NEXT: br label %[[LOOP_HEADER_EXIT_CRIT_EDGE:.*]]
+; WITH-LATE-IPSCCP: [[LOOP_HEADER_EXIT_CRIT_EDGE]]:
+; WITH-LATE-IPSCCP-NEXT: br label %[[EXIT]]
+; WITH-LATE-IPSCCP: [[EXIT]]:
+; WITH-LATE-IPSCCP-NEXT: ret void
+;
+entry:
+ %temp_alloc = alloca i32, align 4
+ br label %loop.header
+
+loop.header:
+ %ptr_input = phi ptr [ %arg1, %entry ], [ %arrayidx, %loop.body ]
+ %iv = phi i32 [ %loop_idx, %entry ], [ %iv.next, %loop.body ]
+ %val = load i32, ptr %ptr_input, align 4
+ %cmp = icmp slt i32 %iv, 1
+ br i1 %cmp, label %loop.body, label %exit
+
+loop.body:
+ %idxprom = sext i32 %iv to i64
+ %arrayidx = getelementptr inbounds i32, ptr %ptr_input, i64 %idxprom
+ store i32 %val, ptr %arrayidx, align 4
+ %iv.next = add nsw i32 %iv, 1
+ call void @digits_2(ptr %temp_alloc, i32 %iv.next)
+ br label %loop.header
+
+exit:
+ ret void
+}
>From a806225fd0011f205da8517199b029681fd6fb67 Mon Sep 17 00:00:00 2001
From: "anoop.kumar6 at ibm.com" <Anoop.Kumar6 at ibm.com>
Date: Fri, 21 Aug 2026 15:40:58 +0200
Subject: [PATCH 3/4] [Pipeline] Add IPSCCPPass at O3 to fix function
specialization phase ordering
Add a second IPSCCPPass with FunctionSpecialization enabled at O3 non-LTO
directly after buildInlinerPipeline. Disable FuncSpec in the existing IPSCCP
pass at O3 non-LTO to avoid redundant work and the inliner cascading the
pre-inliner clones into a single monolithic function. FuncSpec at O1/O2
is unchanged.
LoopRotate inside buildInlinerPipeline restructures the while-style loop
to do-while, exposing constant propagation opportunities that allow
FunctionSpecialization to produce 7 independent clones of the hot
recursive function in exchange2_r. With the pre-inliner placement,
FuncSpec creates 5 clones which the inliner then cascades into one
surviving specialization.
SPECrate 2017 (SystemZ): exchange2_r -19.86% run time, -8.11% compile
time. No significant change on other benchmarks. Geometric mean run time
-1.11%, compile time 0.00%.
CTMark compile time (SystemZ, instruction count): +0.39% geomean vs
+0.47% when FuncSpec runs in both IPSCCP passes.
---
...funcspec-phase-ordering-loop-rotate-ipsccp.f90 | 2 +-
llvm/lib/Passes/PassBuilderPipelines.cpp | 15 +++++++++------
2 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90 b/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90
index 045ca422ab55c..00f518feb2cad 100644
--- a/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90
+++ b/flang/test/Driver/funcspec-phase-ordering-loop-rotate-ipsccp.f90
@@ -45,4 +45,4 @@ end module brute_force
! PASS-O3-PIPE: Running pass: DeadArgumentEliminationPass on [module]
! PASS-O3-IR: define void @{{.*}}top_level_caller
-! PASS-O3-IR: label %{{.*}}digits_2{{.*}}.specialized.
+! PASS-O3-IR: call fastcc void @{{.*}}digits_2{{.*}}.specialized.
diff --git a/llvm/lib/Passes/PassBuilderPipelines.cpp b/llvm/lib/Passes/PassBuilderPipelines.cpp
index 59235c0a6a672..f640234a3cf77 100644
--- a/llvm/lib/Passes/PassBuilderPipelines.cpp
+++ b/llvm/lib/Passes/PassBuilderPipelines.cpp
@@ -1216,8 +1216,12 @@ PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
// and prior to optimizing globals.
// FIXME: This position in the pipeline hasn't been carefully considered in
// years, it should be re-analyzed.
- MPM.addPass(
- IPSCCPPass(IPSCCPOptions(/*AllowFuncSpec=*/!isLTOPreLink(Phase))));
+ // FuncSpec at O3 non-LTO is deferred to after buildInlinerPipeline
+ // where LoopRotate and SROA expose better specialization opportunities.
+ MPM.addPass(IPSCCPPass(IPSCCPOptions(/*AllowFuncSpec=*/
+ !isLTOPreLink(Phase) &&
+ !(Level == OptimizationLevel::O3 &&
+ Phase == ThinOrFullLTOPhase::None))));
// Attach metadata to indirect call sites indicating the set of functions
// they may target at run-time. This should follow IPSCCP.
@@ -1322,11 +1326,10 @@ PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
else
MPM.addPass(buildInlinerPipeline(Level, Phase));
- // Run a second IPSCCP pass at O3 to resolve function specialization
- // opportunities exposed by loop rotation reference parameters
- // (e.g., Fortran pass-by-reference).
+ // Run IPSCCP with FunctionSpecialization after the inliner pipeline
+ // where LoopRotate and SROA expose better specialization opportunities.
if (Level == OptimizationLevel::O3 && Phase == ThinOrFullLTOPhase::None)
- MPM.addPass(llvm::IPSCCPPass(IPSCCPOptions(/*AllowFuncSpec=*/true)));
+ MPM.addPass(IPSCCPPass(IPSCCPOptions(/*AllowFuncSpec=*/true)));
// Remove any dead arguments exposed by cleanups, constant folding globals,
// and argument promotion.
>From 93ba69252acff752cb4b045c39d6f75d6ce48c9d Mon Sep 17 00:00:00 2001
From: "anoop.kumar6 at ibm.com" <Anoop.Kumar6 at ibm.com>
Date: Fri, 21 Aug 2026 20:54:27 +0200
Subject: [PATCH 4/4] clang-format fix
---
llvm/lib/Passes/PassBuilderPipelines.cpp | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/llvm/lib/Passes/PassBuilderPipelines.cpp b/llvm/lib/Passes/PassBuilderPipelines.cpp
index f640234a3cf77..2aba88e2c1954 100644
--- a/llvm/lib/Passes/PassBuilderPipelines.cpp
+++ b/llvm/lib/Passes/PassBuilderPipelines.cpp
@@ -1219,9 +1219,9 @@ PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
// FuncSpec at O3 non-LTO is deferred to after buildInlinerPipeline
// where LoopRotate and SROA expose better specialization opportunities.
MPM.addPass(IPSCCPPass(IPSCCPOptions(/*AllowFuncSpec=*/
- !isLTOPreLink(Phase) &&
- !(Level == OptimizationLevel::O3 &&
- Phase == ThinOrFullLTOPhase::None))));
+ !isLTOPreLink(Phase) &&
+ !(Level == OptimizationLevel::O3 &&
+ Phase == ThinOrFullLTOPhase::None))));
// Attach metadata to indirect call sites indicating the set of functions
// they may target at run-time. This should follow IPSCCP.
More information about the flang-commits
mailing list