[llvm] [llvm-exegesis] Fix LBR hang on hybrid x86 CPUs (PR #216917)

[Fangrui Song via llvm-commits]

https://github.com/MaskRay created https://github.com/llvm/llvm-project/pull/216917

Hybrid CPUs (Alder Lake and newer) register one PMU per core type: cpu_core
for the P-cores and cpu_atom for the E-cores. X86LbrPerfEvent programs a
P-core encoding of BR_INST_RETIRED.NEAR_TAKEN and cpu_core claims
PERF_TYPE_RAW, so the kernel never schedules the event while we run on an
E-core. perf_event_open succeeds anyway and the event records nothing, so
doReadCounter polled for 160*10s (~27 minutes). This stalls a full
llvm/test run built with LLVM_ENABLE_LIBPFM=ON, because lit.local.cfg
probes llvm-exegesis during test discovery, before any test runs.

Pin to the cpu_core CPU set, intersected with the current affinity mask so
that taskset still wins, and report an error when the intersection is
empty. Cut the poll budget to ~3s as well; the event is disabled before
polling, so a sample is either already in the ring buffer or will never
arrive. Give the lit probe a timeout too.

The non-LBR counters share this root cause but already fail loudly via
PerfCounterNotFullyEnabled, so pinning them is left to a follow-up.

Aided by Claude Opus 5


>From 5e8cbf0aa45b3f720e22171a73ad63e7ad046a4f Mon Sep 17 00:00:00 2001
From: Fangrui Song <i at maskray.me>
Date: Mon, 17 Aug 2026 21:48:11 -0700
Subject: [PATCH] [llvm-exegesis] Fix LBR hang on hybrid x86 CPUs

Hybrid CPUs (Alder Lake and newer) register one PMU per core type: cpu_core
for the P-cores and cpu_atom for the E-cores. X86LbrPerfEvent programs a
P-core encoding of BR_INST_RETIRED.NEAR_TAKEN and cpu_core claims
PERF_TYPE_RAW, so the kernel never schedules the event while we run on an
E-core. perf_event_open succeeds anyway and the event records nothing, so
doReadCounter polled for 160*10s (~27 minutes). This stalls a full
llvm/test run built with LLVM_ENABLE_LIBPFM=ON, because lit.local.cfg
probes llvm-exegesis during test discovery, before any test runs.

Pin to the cpu_core CPU set, intersected with the current affinity mask so
that taskset still wins, and report an error when the intersection is
empty. Cut the poll budget to ~3s as well; the event is disabled before
polling, so a sample is either already in the ring buffer or will never
arrive. Give the lit probe a timeout too.

The non-LBR counters share this root cause but already fail loudly via
PerfCounterNotFullyEnabled, so pinning them is left to a follow-up.

Aided by Claude Opus 5
---
 llvm/test/tools/llvm-exegesis/lit.local.cfg   |  7 ++
 .../llvm-exegesis/lib/X86/X86Counter.cpp      | 81 +++++++++++++++++--
 llvm/tools/llvm-exegesis/lib/X86/X86Counter.h |  3 +
 3 files changed, 86 insertions(+), 5 deletions(-)

diff --git a/llvm/test/tools/llvm-exegesis/lit.local.cfg b/llvm/test/tools/llvm-exegesis/lit.local.cfg
index 89110ed2816cd..4951e587e2f17 100644
--- a/llvm/test/tools/llvm-exegesis/lit.local.cfg
+++ b/llvm/test/tools/llvm-exegesis/lit.local.cfg
@@ -19,13 +19,20 @@ def can_use_perf_counters(mode, extra_options=[]):
         print("could not find llvm-exegesis")
         return False
     try:
+        # This runs during test discovery, so a hung llvm-exegesis would stall
+        # the whole lit invocation before a single test starts. Treat a probe
+        # that takes this long as a probe that failed.
         return_code = subprocess.call(
             [llvm_exegesis_exe, "-mode", mode, "-opcode-name=ADD64rr"]
             + extra_options,
             stdout=subprocess.DEVNULL,
             stderr=subprocess.DEVNULL,
+            timeout=60,
         )
         return return_code == 0
+    except subprocess.TimeoutExpired:
+        print("llvm-exegesis timed out")
+        return False
     except OSError:
         print("could not exec llvm-exegesis")
         return False
diff --git a/llvm/tools/llvm-exegesis/lib/X86/X86Counter.cpp b/llvm/tools/llvm-exegesis/lib/X86/X86Counter.cpp
index 9dc6c764599cd..24327790437c9 100644
--- a/llvm/tools/llvm-exegesis/lib/X86/X86Counter.cpp
+++ b/llvm/tools/llvm-exegesis/lib/X86/X86Counter.cpp
@@ -14,8 +14,12 @@
 // FIXME: Use appropriate wrappers for poll.h and mman.h
 // to support Windows and remove this linux-only guard.
 
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/Twine.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Errc.h"
+#include "llvm/Support/MemoryBuffer.h"
 
 #include <perfmon/perf_event.h>
 #include <perfmon/pfmlib.h>
@@ -29,6 +33,7 @@
 #include <memory>
 
 #include <poll.h>
+#include <sched.h>
 #include <sys/mman.h>
 #include <unistd.h>
 
@@ -44,13 +49,75 @@ static const size_t kDataBufferSize = kBufferPages * getpagesize();
 // the next page, so we allocate one more page.
 static const size_t kMappedBufferSize = (kBufferPages + 1) * getpagesize();
 
+// The event is disabled before we poll, so a sample either is already in the
+// ring buffer -- wakeup_events == 1 leaves POLLIN asserted -- or it will never
+// arrive. One poll is therefore conclusive, and the retries only absorb wake-up
+// latency. This used to be 160 * 10s, which turned a host without working LBR
+// into a 27 minute hang.
+constexpr int kPollTimeoutMs = 1000;
+constexpr int kMaxTimeouts = 2;
+
 // Waits for the LBR perf events.
 static int pollLbrPerfEvent(const int FileDescriptor) {
   struct pollfd PollFd;
   PollFd.fd = FileDescriptor;
   PollFd.events = POLLIN;
   PollFd.revents = 0;
-  return poll(&PollFd, 1 /* num of fds */, 10000 /* timeout in ms */);
+  return poll(&PollFd, 1 /* num of fds */, kPollTimeoutMs);
+}
+
+// Pins the process to the P-cores of a hybrid CPU (Alder Lake and newer).
+//
+// Such CPUs have one PMU per core type -- "cpu_core" for the P-cores and
+// "cpu_atom" for the E-cores -- rather than a single "cpu" PMU. The event below
+// is a P-core encoding and "cpu_core" claims PERF_TYPE_RAW, so the kernel never
+// schedules it while we run on an E-core. perf_event_open() succeeds there all
+// the same, so the event silently records nothing at all and we just poll until
+// we give up. Pinning also keeps a measurement from migrating mid-flight.
+static Error pinToLbrCapableCpus() {
+  // sysfs misreports the size of its files, so read this one as a stream. It
+  // only exists on a hybrid CPU; anywhere else every core can run the event.
+  auto BufferOrErr = MemoryBuffer::getFileAsStream(
+      "/sys/bus/event_source/devices/cpu_core/cpus");
+  if (!BufferOrErr)
+    return Error::success();
+  StringRef Cpus = (*BufferOrErr)->getBuffer().trim();
+
+  cpu_set_t Current;
+  if (sched_getaffinity(0, sizeof(Current), &Current) != 0)
+    return make_error<StringError>("Cannot read the CPU affinity.",
+                                   errc::io_error);
+
+  // A list of ranges, e.g. "0-15" or "0,2,4-7". Intersecting it with the
+  // affinity mask lets an affinity the user set with taskset(1) still win.
+  cpu_set_t PCores;
+  CPU_ZERO(&PCores);
+  SmallVector<StringRef, 4> Ranges;
+  Cpus.split(Ranges, ',');
+  for (StringRef Range : Ranges) {
+    auto [FirstStr, LastStr] = Range.split('-');
+    // A bare "N" is the same as the range "N-N".
+    if (LastStr.empty())
+      LastStr = FirstStr;
+    unsigned First, Last;
+    if (FirstStr.getAsInteger(10, First) || LastStr.getAsInteger(10, Last))
+      return make_error<StringError>("Cannot parse the P-core CPU list '" +
+                                         Cpus + "'",
+                                     errc::invalid_argument);
+    for (unsigned I = First; I <= Last && I < CPU_SETSIZE; ++I)
+      if (CPU_ISSET(I, &Current))
+        CPU_SET(I, &PCores);
+  }
+
+  if (CPU_COUNT(&PCores) == 0)
+    return make_error<StringError>(
+        "LBR measurements have to run on a P-core, but the CPU affinity mask "
+        "of this process selects E-cores only.",
+        errc::not_supported);
+  if (sched_setaffinity(0, sizeof(PCores), &PCores) != 0)
+    return make_error<StringError>("Cannot pin the process to the P-cores.",
+                                   errc::io_error);
+  return Error::success();
 }
 
 // Copies the data-buffer into Buf, given the pointer to MMapped.
@@ -122,7 +189,9 @@ X86LbrPerfEvent::X86LbrPerfEvent(unsigned SamplingPeriod) {
   Attr = new perf_event_attr();
   Attr->size = sizeof(*Attr);
   Attr->type = PERF_TYPE_RAW;
-  // FIXME This is SKL's encoding. Not sure if it'll change.
+  // FIXME This is SKL's encoding. Not sure if it'll change. It is also a
+  // P-core encoding, so on a hybrid CPU this only counts while running on a
+  // P-core; see pinToLbrCapableCpus().
   Attr->config = 0x20c4; // BR_INST_RETIRED.NEAR_TAKEN
   Attr->sample_type = PERF_SAMPLE_BRANCH_STACK;
   // Don't need to specify "USER" because we've already excluded HV and Kernel.
@@ -155,6 +224,11 @@ void X86LbrCounter::start() {
 }
 
 Error X86LbrCounter::checkLbrSupport() {
+  // Make sure we do not run on an E-core, where the event would silently
+  // record nothing at all.
+  if (Error E = pinToLbrCapableCpus())
+    return E;
+
   // Do a sample read and check if the results contain non-zero values.
 
   X86LbrCounter counter(X86LbrPerfEvent(123));
@@ -209,9 +283,6 @@ X86LbrCounter::readOrError(StringRef FunctionBytes) const {
 
 Expected<SmallVector<int64_t, 4>>
 X86LbrCounter::doReadCounter(const void *From, const void *To) const {
-  // The max number of time-outs/retries before we give up.
-  static constexpr int kMaxTimeouts = 160;
-
   // Parses the LBR buffer and fills CycleArray with the sequence of cycle
   // counts from the buffer.
   SmallVector<int64_t, 4> CycleArray;
diff --git a/llvm/tools/llvm-exegesis/lib/X86/X86Counter.h b/llvm/tools/llvm-exegesis/lib/X86/X86Counter.h
index 75f687d395dec..527a3e957f404 100644
--- a/llvm/tools/llvm-exegesis/lib/X86/X86Counter.h
+++ b/llvm/tools/llvm-exegesis/lib/X86/X86Counter.h
@@ -33,6 +33,9 @@ class X86LbrPerfEvent : public pfm::PerfEvent {
 
 class X86LbrCounter : public pfm::CounterGroup {
 public:
+  // Checks whether the host supports LBR with cycles. On a hybrid CPU this also
+  // pins the process to the P-cores, and that pinning persists so that the
+  // later measurement runs stay on a CPU where the event is scheduled.
   static Error checkLbrSupport();
 
   explicit X86LbrCounter(pfm::PerfEvent &&Event);