[llvm-branch-commits] [llvm] [llvm-profgen] Infer callsite samples from callee head samples (PR #235988)
Kunal Pathak via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Wed Nov 18 14:09:05 PST 2026
================
@@ -1377,6 +1386,96 @@ void CSProfileGenerator::populateBoundarySamplesWithProbes(
}
}
+void CSProfileGenerator::inferCallsiteSamples() {
+ // Walk the context trie bottom-up (children before parents) and for each
+ // node, ensure that callsite body samples are at least as large as callee
+ // entry (head) samples. This serves two purposes:
+ // 1. Create FunctionSamples for nodes without samples and infer called
+ // target samples using children's head samples.
+ // 2. Fix under-counted callsites for nodes that have callsite counts
+ // lower than callee entry counts.
+ SmallVector<ContextTrieNode *, 256> AllNodes;
+ for (auto *Node : ContextTracker)
+ AllNodes.push_back(Node);
+
+ for (auto It = AllNodes.rbegin(), End = AllNodes.rend(); It != End; ++It) {
+ ContextTrieNode *Node = *It;
+
+ if (Node == &getRootContext())
+ continue;
+
+ bool HasChildWithSamples = false;
+ for (auto &Child : Node->getAllChildContext()) {
+ if (Child.second.getFunctionSamples()) {
+ HasChildWithSamples = true;
+ break;
+ }
+ }
+ if (!HasChildWithSamples)
+ continue;
+
+ FunctionSamples *FSamples = Node->getFunctionSamples();
+ if (!FSamples) {
+ FSamples = getOrCreateFunctionSamples(Node);
+ const auto *FuncDesc =
+ Binary->getFuncDescForGUID(Node->getFuncName().getHashCode());
+ FSamples->setFunctionHash(FuncDesc->FuncHash);
+ }
+
+ // Sum the children's samples per callsite: each child's head samples, or
+ // its recorded called target if that is larger.
+ std::map<LineLocation, uint64_t> CallsiteHeadSum;
+ for (auto &Child : Node->getAllChildContext()) {
+ FunctionSamples *ChildSamples = Child.second.getFunctionSamples();
+ if (!ChildSamples)
+ continue;
+ uint64_t ChildHeadSamples = ChildSamples->getHeadSamplesEstimate();
----------------
kunalspathak wrote:
I ran your test input (`infer-callsite-samples-pseudoprobe.perfscript/.perfbin`) through a build of this patch, and I wanted to double check the head estimate functioning. All 16 LBR entries in that sample are the same branch, 0x40124d -> 0x4011c0, which is work's loop back-edge. So every count in the profile is loop iterations, and work has no head samples.
With `--infer-callsite-samples=false --csspgo-preinliner=0 --gen-cs-nested-profile=0` there's only work's context (`.__uniq.*` suffixes trimmed):
```
[main:5 @ top:5 @ wrapper:3 @ top:5 @ wrapper:5 @ work]:480:0
2: 120
3: 0
4: 120
7: 120
8: 120
9: 0
```
Head is 0, so `getHeadSamplesEstimate()` falls back to the first body record i..e probe 2, inside the loop, 120. With inference on, that 120 is copied to every callsite up the chain:
```
[main]:120:0
5: 120
[main:5 @ top]:120:0
5: 120
[main:5 @ top:5 @ wrapper]:120:0
3: 120
[main:5 @ top:5 @ wrapper:3 @ top]:120:0
5: 120
[main:5 @ top:5 @ wrapper:3 @ top:5 @ wrapper]:120:0
5: 120 work:120
```
And with the preinliner on, infer's off vs on looks like this:
```
infer off: infer on:
work:480:0 top:840:0
2: 120 5: 240
... 5: wrapper:600
3: 120
5: work:480
...
main:120:0
5: 120
```
Keeping the `top -> wrapper -> work` chain is what we get with this PR, and that part looks good. But main calls top 4 times in the source, and here main's callsite has the same count as work's inner loop (and top's is 240). The created profiles feed it up the chain too. Their own estimate is again their first body record, i.e. the number we just put there. Since the summary is computed after this runs (`postProcessProfiles -> computeSummaryAndThreshold`), it also moves the hot/cold thresholds, and Pass2 will see main's loop block as hot as work's loop.
For the cases in the description (zero-size wrappers, always_inline) the count doesn't matter, they get inlined anyway. Could the raise use real head samples (`getHeadSamples()`) and skip the estimate? It can still create the profiles for the chain, just without turning loop iterations into call counts.
https://github.com/llvm/llvm-project/pull/235988
More information about the llvm-branch-commits
mailing list