[llvm] 511500e - [TypeProf][ICP]Allow vtable-comparison as long as vtable count is comparable with function count for each candidate (#106260)

[via llvm-commits]

Author: Mingming Liu
Date: 2024-08-27T11:51:24-07:00
New Revision: 511500e35159fb770f6a570f473beeb78d4091a4

URL: https://github.com/llvm/llvm-project/commit/511500e35159fb770f6a570f473beeb78d4091a4
DIFF: https://github.com/llvm/llvm-project/commit/511500e35159fb770f6a570f473beeb78d4091a4.diff

LOG: [TypeProf][ICP]Allow vtable-comparison as long as vtable count is comparable with function count for each candidate (#106260)

The current cost-benefit analysis between vtable comparison and function
comparison require the indirect fallback branch to be cold. This is too
conservative.

This change allows vtable-comparison as long as vtable count is
comparable with function count for each function candidate and removes
the cold indirect fallback requirement.

Tested:
1. Testing this on benchmarks uplifts the measurable performance wins.
Counting the (possibly-duplicated) remarks (because of linkonce_odr
functions, cross-module import of functions) show the number of vtable
remarks increases from ~30k-ish to 50k-ish.
2. https://gcc.godbolt.org/z/sbGK7Pacn shows vtable-comparison doesn't
happen today (using the same IR input)

Added: 
    

Modified: 
    llvm/lib/Transforms/Instrumentation/IndirectCallPromotion.cpp
    llvm/test/Transforms/PGOProfile/icp_vtable_cmp.ll

Removed: 
    


################################################################################
diff  --git a/llvm/lib/Transforms/Instrumentation/IndirectCallPromotion.cpp b/llvm/lib/Transforms/Instrumentation/IndirectCallPromotion.cpp
index 0d1f506986379d..c9007498ee2331 100644
--- a/llvm/lib/Transforms/Instrumentation/IndirectCallPromotion.cpp
+++ b/llvm/lib/Transforms/Instrumentation/IndirectCallPromotion.cpp
@@ -117,7 +117,7 @@ static cl::opt<bool>
 // Indirect call promotion pass will fall back to function-based comparison if
 // vtable-count / function-count is smaller than this threshold.
 static cl::opt<float> ICPVTablePercentageThreshold(
-    "icp-vtable-percentage-threshold", cl::init(0.99), cl::Hidden,
+    "icp-vtable-percentage-threshold", cl::init(0.995), cl::Hidden,
     cl::desc("The percentage threshold of vtable-count / function-count for "
              "cost-benefit analysis."));
 
@@ -888,14 +888,6 @@ bool IndirectCallPromoter::isProfitableToCompareVTables(
     }
   }
 
-  // If the indirect fallback is not cold, don't compare vtables.
-  if (PSI && PSI->hasProfileSummary() &&
-      !PSI->isColdCount(RemainingVTableCount)) {
-    LLVM_DEBUG(dbgs() << "    Indirect fallback basic block is not cold. Bail "
-                         "out for vtable comparison.\n");
-    return false;
-  }
-
   return true;
 }
 

diff  --git a/llvm/test/Transforms/PGOProfile/icp_vtable_cmp.ll b/llvm/test/Transforms/PGOProfile/icp_vtable_cmp.ll
index c77be3b1ed2444..b6afce3d7c6d5d 100644
--- a/llvm/test/Transforms/PGOProfile/icp_vtable_cmp.ll
+++ b/llvm/test/Transforms/PGOProfile/icp_vtable_cmp.ll
@@ -120,6 +120,7 @@ declare i32 @Base2_foo(ptr)
 declare i32 @Base1_foo(ptr)
 declare void @Base3_foo(ptr)
 
+!llvm.module.flags = !{!11}
 !0 = !{i64 16, !"Base1"}
 !1 = !{i64 40, !"Base1"}
 !2 = !{i64 16, !"Base2"}
@@ -131,6 +132,23 @@ declare void @Base3_foo(ptr)
 !8 = !{i64 16, !"Derived3"}
 !9 = !{!"VP", i32 2, i64 1600, i64 -4123858694673519054, i64 600, i64 -7211198353767973908, i64 500, i64 -3574436251470806727, i64 200, i64 6288809125658696740, i64 200, i64 12345678, i64 100}
 !10 = !{!"VP", i32 0, i64 1600, i64 3827408714133779784, i64 600, i64 5837445539218476403, i64 500, i64 -9064955852395570538, i64 400,  i64 56781234, i64 100}
+
+!11 = !{i32 1, !"ProfileSummary", !12}
+!12 = !{!13, !14, !15, !16, !17, !18, !19, !20}
+!13 = !{!"ProfileFormat", !"InstrProf"}
+!14 = !{!"TotalCount", i64 10000}
+!15 = !{!"MaxCount", i64 10}
+!16 = !{!"MaxInternalCount", i64 1}
+!17 = !{!"MaxFunctionCount", i64 1000}
+!18 = !{!"NumCounts", i64 3}
+!19 = !{!"NumFunctions", i64 3}
+!20 = !{!"DetailedSummary", !21}
+!21 = !{!22, !23, !24}
+!22 = !{i32 10000, i64 101, i32 1}
+!23 = !{i32 990000, i64 101, i32 1}
+!24 = !{i32 999999, i64 1, i32 2}
+
+
 ;.
 ; VTABLE-COMMON: [[PROF9]] = !{!"VP", i32 2, i64 100, i64 12345678, i64 100}
 ; VTABLE-COMMON: [[PROF10]] = !{!"branch_weights", i32 600, i32 1000}