[PATCH] D108677: [CSSPGO] Add switch for sample loader to honor global pre-inliner decision from llvm-profgen
Wenlei He via Phabricator via llvm-commits
llvm-commits at lists.llvm.org
Tue Aug 24 17:35:53 PDT 2021
wenlei created this revision.
wenlei added reviewers: hoy, wmi, wlei.
Herald added subscribers: ormris, modimo, lxfind, hiraditya.
wenlei requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.
The change adds a switch to allow sample loader to use global pre-inliner's decision instead. The pre-inliner in llvm-profgen makes inline decision globally based on whole program profile and function byte size as cost proxy.
Since pre-inliner also adjusts/merges context profile based on its inline decision, honoring its inline decision in sample loader would lead to better post-inline profile quality especially for thinlto where cross module profile merging isn't possible without pre-inliner.
Minor fix in profile reader is also included. When pre-inliner is use, we now also turn off the default merging and trimming logic unless it's explicitly asked.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D108677
Files:
llvm/lib/ProfileData/SampleProfReader.cpp
llvm/lib/Transforms/IPO/SampleProfile.cpp
llvm/tools/llvm-profgen/ProfileGenerator.cpp
Index: llvm/tools/llvm-profgen/ProfileGenerator.cpp
===================================================================
--- llvm/tools/llvm-profgen/ProfileGenerator.cpp
+++ llvm/tools/llvm-profgen/ProfileGenerator.cpp
@@ -407,11 +407,15 @@
.run();
}
- // Trim and merge cold context profile using cold threshold above;
- SampleContextTrimmer(ProfileMap)
- .trimAndMergeColdContextProfiles(
- ColdCountThreshold, CSProfTrimColdContext, CSProfMergeColdContext,
- CSProfMaxColdContextDepth);
+ // Trim and merge cold context profile using cold threshold above. By default,
+ // we skip such merging and trimming when preinliner is on.
+ if (!EnableCSPreInliner || CSProfTrimColdContext.getNumOccurrences() ||
+ CSProfMergeColdContext.getNumOccurrences()) {
+ SampleContextTrimmer(ProfileMap)
+ .trimAndMergeColdContextProfiles(
+ ColdCountThreshold, CSProfTrimColdContext, CSProfMergeColdContext,
+ CSProfMaxColdContextDepth);
+ }
}
void CSProfileGenerator::computeSummaryAndThreshold() {
Index: llvm/lib/Transforms/IPO/SampleProfile.cpp
===================================================================
--- llvm/lib/Transforms/IPO/SampleProfile.cpp
+++ llvm/lib/Transforms/IPO/SampleProfile.cpp
@@ -214,6 +214,11 @@
cl::desc("Use call site prioritized inlining for sample profile loader."
"Currently only CSSPGO is supported."));
+static cl::opt<bool> UsePreInlinerDecision(
+ "sample-profile-use-preinliner", cl::Hidden, cl::ZeroOrMore,
+ cl::init(false),
+ cl::desc("Use the preinliner decisions stored in profile context."));
+
static cl::opt<std::string> ProfileInlineReplayFile(
"sample-profile-inline-replay", cl::init(""), cl::value_desc("filename"),
cl::desc(
@@ -1257,6 +1262,21 @@
return InlineCost::getAlways("previously inlined");
}
+ // With CSSPGO, the preinliner in llvm-profgen can estimate global inline
+ // decisions based on hotness as well as accurate function byte sizes for
+ // given context using function/inlinee sizes from previous build. It
+ // stores the decision in profile, and also adjust/merge context profile
+ // aiming at better context-sensitive post-inline profile quality, assming
+ // all inline decision estimates are going to be honored by compiler. Here
+ // we replay that inline decision under `sample-profile-use-preinliner`.
+ if (UsePreInlinerDecision) {
+ if (Candidate.CalleeSamples->getContext().hasAttribute(
+ ContextShouldBeInlined))
+ return InlineCost::getAlways("preinliner");
+ else
+ return InlineCost::getNever("preinliner");
+ }
+
// Adjust threshold based on call site hotness, only do this for callsite
// prioritized inliner because otherwise cost-benefit check is done earlier.
int SampleThreshold = SampleColdCallSiteThreshold;
Index: llvm/lib/ProfileData/SampleProfReader.cpp
===================================================================
--- llvm/lib/ProfileData/SampleProfReader.cpp
+++ llvm/lib/ProfileData/SampleProfReader.cpp
@@ -791,7 +791,7 @@
}
assert((CSProfileCount == 0 || CSProfileCount == Profiles.size()) &&
"Cannot have both context-sensitive and regular profile");
- assert(ProfileIsCS == (CSProfileCount > 0) &&
+ assert((!CSProfileCount || ProfileIsCS == (CSProfileCount > 0)) &&
"Section flag should be consistent with actual profile");
return sampleprof_error::success;
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D108677.368509.patch
Type: text/x-patch
Size: 3477 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210825/653004ac/attachment.bin>
More information about the llvm-commits
mailing list