[PATCH] D151614: [BOLT] Align BranchInfo and FuncBranchData in DataAggregator::recordTrace

[Amir Ayupov via Phabricator via llvm-commits]

Amir created this revision.
Amir added a reviewer: bolt.
Herald added a reviewer: rafauler.
Herald added subscribers: treapster, ayermolo.
Herald added a reviewer: maksfb.
Herald added a project: All.
Amir requested review of this revision.
Herald added subscribers: llvm-commits, yota9.
Herald added a project: LLVM.

`DataAggregator::recordTrace` serves two purposes:

- Attaching LBR fallthrough ("trace") information to CFG (`getBranchInfo`), which eventually gets emitted as YAML profile.
- Populating vector of offsets that gets added to `FuncBranchData`, which eventually gets emitted as fdata profile.

`recordTrace` is invoked from `getFallthroughsInTrace` which checks its return
status and passes on the collected vector of offsets to `doTrace`.

However, if a malformed trace is passed to `recordTrace` it might partially
attach the profile to CFG and exit with false, not propagating the vector of
offsets to `doTrace`. This leads to a difference between fdata and yaml profile
collected from the same binary and the same perf file.

(Skylake LBR errata might produce such malformed traces where the last entry
is duplicated, resulting in invalid fallthrough path between the last two
entries).

There are two ways to handle this mismatch: conservative (aligned with fdata),
or aggressive (aligned with yaml). Conservative approach would discard the
trace entirely, buffering the CFG updates until all fallthroughs are confirmed.
Aggressive approach would apply CFG updates and return the matching
fallthroughs in the vector even if the trace is invalid (doesn't correspond to
a valid fallthrough path). I chose to go with the former (conservative/fdata)
approach which produces more accurate profile.

We can't rely on pre-filtering such traces early (in LBR sample processing) as 
DataAggregator is used for both perf samples and pre-aggregated perf information
which loses branch stack information.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D151614

Files:
  bolt/lib/Profile/DataAggregator.cpp


Index: bolt/lib/Profile/DataAggregator.cpp
===================================================================
--- bolt/lib/Profile/DataAggregator.cpp
+++ bolt/lib/Profile/DataAggregator.cpp
@@ -902,10 +902,6 @@
       return false;
     }
 
-    // Record fall-through jumps
-    BinaryBasicBlock::BinaryBranchInfo &BI = BB->getBranchInfo(*NextBB);
-    BI.Count += Count;
-
     if (Branches) {
       const MCInst *Instr = BB->getLastNonPseudoInstr();
       uint64_t Offset = 0;
@@ -920,6 +916,15 @@
     BB = NextBB;
   }
 
+  // Record fall-through jumps
+  for (const auto &[FromOffset, ToOffset] : *Branches) {
+    BinaryBasicBlock *FromBB = BF.getBasicBlockContainingOffset(FromOffset);
+    BinaryBasicBlock *ToBB = BF.getBasicBlockAtOffset(ToOffset);
+    assert(FromBB && ToBB);
+    BinaryBasicBlock::BinaryBranchInfo &BI = FromBB->getBranchInfo(*ToBB);
+    BI.Count += Count;
+  }
+
   return true;
 }
 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D151614.526274.patch
Type: text/x-patch
Size: 920 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230527/480a6e9e/attachment.bin>