[PATCH] D108677: [CSSPGO] Add switch for sample loader to honor global pre-inliner decision from llvm-profgen

[Wenlei He via Phabricator via llvm-commits]

wenlei created this revision.
wenlei added reviewers: hoy, wmi, wlei.
Herald added subscribers: ormris, modimo, lxfind, hiraditya.
wenlei requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

The change adds a switch to allow sample loader to use global pre-inliner's decision instead. The pre-inliner in llvm-profgen makes inline decision globally based on whole program profile and function byte size as cost proxy.

Since pre-inliner also adjusts/merges context profile based on its inline decision, honoring its inline decision in sample loader would lead to better post-inline profile quality especially for thinlto where cross module profile merging isn't possible without pre-inliner.

Minor fix in profile reader is also included. When pre-inliner is use, we now also turn off the default merging and trimming logic unless it's explicitly asked.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D108677

Files:
  llvm/lib/ProfileData/SampleProfReader.cpp
  llvm/lib/Transforms/IPO/SampleProfile.cpp
  llvm/tools/llvm-profgen/ProfileGenerator.cpp


Index: llvm/tools/llvm-profgen/ProfileGenerator.cpp
===================================================================
--- llvm/tools/llvm-profgen/ProfileGenerator.cpp
+++ llvm/tools/llvm-profgen/ProfileGenerator.cpp
@@ -407,11 +407,15 @@
         .run();
   }
 
-  // Trim and merge cold context profile using cold threshold above;
-  SampleContextTrimmer(ProfileMap)
-      .trimAndMergeColdContextProfiles(
-          ColdCountThreshold, CSProfTrimColdContext, CSProfMergeColdContext,
-          CSProfMaxColdContextDepth);
+  // Trim and merge cold context profile using cold threshold above. By default,
+  // we skip such merging and trimming when preinliner is on.
+  if (!EnableCSPreInliner || CSProfTrimColdContext.getNumOccurrences() ||
+      CSProfMergeColdContext.getNumOccurrences()) {
+    SampleContextTrimmer(ProfileMap)
+        .trimAndMergeColdContextProfiles(
+            ColdCountThreshold, CSProfTrimColdContext, CSProfMergeColdContext,
+            CSProfMaxColdContextDepth);
+  }
 }
 
 void CSProfileGenerator::computeSummaryAndThreshold() {
Index: llvm/lib/Transforms/IPO/SampleProfile.cpp
===================================================================
--- llvm/lib/Transforms/IPO/SampleProfile.cpp
+++ llvm/lib/Transforms/IPO/SampleProfile.cpp
@@ -214,6 +214,11 @@
     cl::desc("Use call site prioritized inlining for sample profile loader."
              "Currently only CSSPGO is supported."));
 
+static cl::opt<bool> UsePreInlinerDecision(
+    "sample-profile-use-preinliner", cl::Hidden, cl::ZeroOrMore,
+    cl::init(false),
+    cl::desc("Use the preinliner decisions stored in profile context."));
+
 static cl::opt<std::string> ProfileInlineReplayFile(
     "sample-profile-inline-replay", cl::init(""), cl::value_desc("filename"),
     cl::desc(
@@ -1257,6 +1262,21 @@
     return InlineCost::getAlways("previously inlined");
   }
 
+  // With CSSPGO, the preinliner in llvm-profgen can estimate global inline
+  // decisions based on hotness as well as accurate function byte sizes for
+  // given context using function/inlinee sizes from previous build. It
+  // stores the decision in profile, and also adjust/merge context profile
+  // aiming at better context-sensitive post-inline profile quality, assming
+  // all inline decision estimates are going to be honored by compiler. Here
+  // we replay that inline decision under `sample-profile-use-preinliner`.
+  if (UsePreInlinerDecision) {
+    if (Candidate.CalleeSamples->getContext().hasAttribute(
+            ContextShouldBeInlined))
+      return InlineCost::getAlways("preinliner");
+    else
+      return InlineCost::getNever("preinliner");
+  }
+
   // Adjust threshold based on call site hotness, only do this for callsite
   // prioritized inliner because otherwise cost-benefit check is done earlier.
   int SampleThreshold = SampleColdCallSiteThreshold;
Index: llvm/lib/ProfileData/SampleProfReader.cpp
===================================================================
--- llvm/lib/ProfileData/SampleProfReader.cpp
+++ llvm/lib/ProfileData/SampleProfReader.cpp
@@ -791,7 +791,7 @@
   }
   assert((CSProfileCount == 0 || CSProfileCount == Profiles.size()) &&
          "Cannot have both context-sensitive and regular profile");
-  assert(ProfileIsCS == (CSProfileCount > 0) &&
+  assert((!CSProfileCount || ProfileIsCS == (CSProfileCount > 0)) &&
          "Section flag should be consistent with actual profile");
   return sampleprof_error::success;
 }


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D108677.368509.patch
Type: text/x-patch
Size: 3477 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210825/653004ac/attachment.bin>