[llvm] Add minimum count threshold for indirect call promotion (PR #145282)

[Snehasish Kumar via llvm-commits]

https://github.com/snehasish updated https://github.com/llvm/llvm-project/pull/145282

>From 592ed4be1d9a83f23a176f711ad5f482de49aa2e Mon Sep 17 00:00:00 2001
From: Snehasish Kumar <snehasishk at google.com>
Date: Sun, 22 Jun 2025 23:45:19 -0700
Subject: [PATCH 1/2] Add minimum count threshold for indirect call promotion

This adds a new command-line option -icp-minimum-count-threshold to prevent
promotion of indirect call candidates with very low absolute counts, even
if they meet the percentage-based profitability thresholds.

Currently, candidates with counts as low as 1-2 calls can be promoted if
they represent a significant percentage of the call site. This can lead to
unprofitable micro-optimizations that increase code size without meaningful
performance benefits.

The new threshold defaults to 0 to maintain backward compatibility but can
be set to values like 5-20 for production use to filter out statistically
insignificant candidates.

Added comprehensive test coverage for the new threshold including negative
tests (blocking low-count promotions), positive tests (allowing high-count
promotions), and edge cases (threshold exactly at count value).
---
 .../IndirectCallPromotionAnalysis.cpp         |  9 +++++++-
 .../PGOProfile/indirect_call_promotion.ll     | 21 +++++++++++++++++++
 2 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp b/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp
index 90065e053b629..861dc0d364b84 100644
--- a/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp
+++ b/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp
@@ -37,6 +37,12 @@ static cl::opt<unsigned>
                              cl::desc("The percentage threshold against total "
                                       "count for the promotion"));
 
+// Set the minimum absolute count threshold for indirect call promotion.
+// Candidates with counts below this threshold will not be promoted.
+static cl::opt<unsigned> ICPMinimumCountThreshold(
+    "icp-minimum-count-threshold", cl::init(0), cl::Hidden,
+    cl::desc("Minimum absolute count for promotion candidate"));
+
 // Set the maximum number of targets to promote for a single indirect-call
 // callsite.
 static cl::opt<unsigned>
@@ -51,7 +57,8 @@ cl::opt<unsigned> MaxNumVTableAnnotations(
 bool ICallPromotionAnalysis::isPromotionProfitable(uint64_t Count,
                                                    uint64_t TotalCount,
                                                    uint64_t RemainingCount) {
-  return Count * 100 >= ICPRemainingPercentThreshold * RemainingCount &&
+  return Count >= ICPMinimumCountThreshold &&
+         Count * 100 >= ICPRemainingPercentThreshold * RemainingCount &&
          Count * 100 >= ICPTotalPercentThreshold * TotalCount;
 }
 
diff --git a/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll b/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
index 3b0e3250cd6df..7f1fea9d1386a 100644
--- a/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
+++ b/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
@@ -1,6 +1,12 @@
 ; RUN: opt < %s -passes=pgo-icall-prom -S -icp-total-percent-threshold=50 | FileCheck %s --check-prefix=ICALL-PROM
 ; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS-REMARK
 ; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=20 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS2-REMARK
+; Test minimum count threshold - should prevent func1 promotion (count 10 < threshold 15)
+; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=15 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-BLOCK
+; Test minimum count threshold - should allow func4 promotion (count 1030 > threshold 15) 
+; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=15 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-ALLOW
+; Test edge case - threshold exactly at count value
+; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=10 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-EDGE
 
 ; PASS-REMARK: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
 ; PASS-REMARK: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
@@ -12,6 +18,21 @@
 ; PASS2-REMARK-NOT: remark: <unknown>:0:0: Promote indirect call to func3
 ; PASS2-REMARK-NOT: remark: <unknown>:0:0: Promote indirect call to func1
 
+; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
+; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
+; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
+; MIN-COUNT-BLOCK-NOT: remark: <unknown>:0:0: Promote indirect call to func1
+
+; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
+; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
+; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
+; MIN-COUNT-ALLOW-NOT: remark: <unknown>:0:0: Promote indirect call to func1
+
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func1 with count 10 out of 10
+
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 

>From 5b0b5ff58a53e2f7c3ee29bea39d536b80f95688 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar <snehasishk at google.com>
Date: Thu, 26 Jun 2025 17:23:57 +0000
Subject: [PATCH 2/2] Drop redundant test case, MIN-COUNT-BLOCK.

---
 llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll b/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
index 7f1fea9d1386a..4f08cb11296b6 100644
--- a/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
+++ b/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
@@ -2,8 +2,6 @@
 ; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS-REMARK
 ; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=20 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS2-REMARK
 ; Test minimum count threshold - should prevent func1 promotion (count 10 < threshold 15)
-; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=15 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-BLOCK
-; Test minimum count threshold - should allow func4 promotion (count 1030 > threshold 15) 
 ; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=15 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-ALLOW
 ; Test edge case - threshold exactly at count value
 ; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=10 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-EDGE
@@ -18,11 +16,6 @@
 ; PASS2-REMARK-NOT: remark: <unknown>:0:0: Promote indirect call to func3
 ; PASS2-REMARK-NOT: remark: <unknown>:0:0: Promote indirect call to func1
 
-; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
-; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
-; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
-; MIN-COUNT-BLOCK-NOT: remark: <unknown>:0:0: Promote indirect call to func1
-
 ; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
 ; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
 ; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160