[llvm] [llvm-profgen] Infer callsite samples for functions without samples (PR #191572)

Wei Wang via llvm-commits llvm-commits at lists.llvm.org
Wed Apr 29 17:26:36 PDT 2026


================
@@ -1373,6 +1382,102 @@ void CSProfileGenerator::populateBoundarySamplesWithProbes(
   }
 }
 
+void CSProfileGenerator::inferCallsiteSamples() {
+  // Walk the context trie bottom-up and for each node, ensure that callsite
+  // body samples are at least as large as callee entry (head) samples. This
+  // handles three cases:
+  //
+  // 1. Probeless functions: inlined functions fully optimized away (e.g., thin
+  //    wrappers) that have no probes in the binary.
+  //
+  // 2. Cold functions: functions that have probes in the binary but no samples
+  //    landed on them.
+  //
+  // 3. Under-counted callsites: nodes that already have samples but callsite
+  //    counts is lower than callee entry counts.
+  //
+  // Note: cases 1 and 2 are not distinguished here — both result in a missing
+  // FunctionSamples from the trie node, but the pre-inliner separates them via
+  // size: probeless functions get size 0 from size tracker, while cold
+  // functions keep their real (non-zero) size.
+  SmallVector<ContextTrieNode *, 256> AllNodes;
+  for (auto *Node : ContextTracker)
+    AllNodes.push_back(Node);
+
+  for (auto It = AllNodes.rbegin(), End = AllNodes.rend(); It != End; ++It) {
+    ContextTrieNode *Node = *It;
+
+    if (Node == &getRootContext())
+      continue;
+
+    bool HasChildWithSamples = false;
+    for (auto &Child : Node->getAllChildContext()) {
+      if (Child.second.getFunctionSamples()) {
+        HasChildWithSamples = true;
+        break;
+      }
+    }
+    if (!HasChildWithSamples)
+      continue;
+
+    // Create FunctionSamples for nodes without them (probeless or cold).
+    FunctionSamples *FSamples = Node->getFunctionSamples();
+    if (!FSamples) {
+      FSamples = getOrCreateFunctionSamples(Node);
+      const auto *FuncDesc =
+          Binary->getFuncDescForGUID(Node->getFuncName().getHashCode());
+      FSamples->setFunctionHash(FuncDesc->FuncHash);
+    }
+
+    // Aggregate callee head samples per callsite, since multiple callees
+    // (e.g., indirect calls) can share the same callsite location. Also add
+    // each callee as a called target.
+    std::map<LineLocation, uint64_t> CallsiteHeadSum;
+    for (auto &Child : Node->getAllChildContext()) {
+      FunctionSamples *ChildSamples = Child.second.getFunctionSamples();
+      if (!ChildSamples)
+        continue;
+      uint64_t ChildHeadSamples = ChildSamples->getHeadSamplesEstimate();
+      if (ChildHeadSamples == 0)
+        continue;
+      LineLocation ChildCallsite = Child.second.getCallSiteLoc();
+      // Only add the delta to avoid double-counting if there are already
+      // called target samples from normal profile generation.
+      uint64_t ExistingTargetSamples = 0;
+      auto ExistingTargets = FSamples->findCallTargetMapAt(
+          ChildCallsite.LineOffset, ChildCallsite.Discriminator);
+      if (ExistingTargets) {
+        auto TargetIt = ExistingTargets->find(Child.second.getFuncName());
+        if (TargetIt != ExistingTargets->end())
+          ExistingTargetSamples = TargetIt->second;
+      }
+      // Use the max of head samples and existing target samples to compute
+      // the expected contribution of this callee to the callsite total.
+      CallsiteHeadSum[ChildCallsite] +=
+          std::max(ChildHeadSamples, ExistingTargetSamples);
+      if (ChildHeadSamples > ExistingTargetSamples)
+        FSamples->addCalledTargetSamples(
+            ChildCallsite.LineOffset, ChildCallsite.Discriminator,
+            Child.second.getFuncName(),
+            ChildHeadSamples - ExistingTargetSamples);
+    }
+
+    // Ensure callsite body samples >= sum of callee head samples.
+    for (auto &[Loc, HeadSum] : CallsiteHeadSum) {
+      uint64_t ExistingSamples = 0;
+      auto Existing =
+          FSamples->findSamplesAt(Loc.LineOffset, Loc.Discriminator);
+      if (Existing)
+        ExistingSamples = *Existing;
+      if (HeadSum > ExistingSamples) {
+        uint64_t Delta = HeadSum - ExistingSamples;
+        FSamples->addBodySamples(Loc.LineOffset, Loc.Discriminator, Delta);
+        FSamples->addTotalSamples(Delta);
+      }
----------------
apolloww wrote:

Yeah, they look similar, but I found `updateFunctionSamples` is not consistently called among different types of profiles.

There are 2 variances of it: 
* `ProfileGeneratorBase::updateFunctionSamples()` is called by non-CS profiles, both line number based and probe based.
* `CSProfileGenerator::updateFunctionSamples()` is only called by CS line number based profile, but not CSSPGO.

Also there is knob inside these functions (`UpdateTotalSamples`) that is set to false by default. I am not sure the historical reason behind this.

https://github.com/llvm/llvm-project/pull/191572


More information about the llvm-commits mailing list