[llvm] [llvm-exegesis] Error Out If Perf Counter is Not Fully Enabled (PR #132892)

[Aiden Grossman via llvm-commits]

https://github.com/boomanaiden154 created https://github.com/llvm/llvm-project/pull/132892

Perf counters can be multiplexed if there are too many that need to be scheduled on a core at the same time (and they exceed the available PMUs). Other processes (especially system ones in certain environments, not commonly on Desktop Linux from what I've seen) can also interfere. This will impact the measurement fidelity as the counter is not actually counting cycles/uops the entire time. This patch makes it so that we error out in these cases so the user gets a visible indication things have gone wrong rather than things failing silently.

>From 1998b3d0b4822b3054d486f76f9d67c6d385c8d0 Mon Sep 17 00:00:00 2001
From: Aiden Grossman <aidengrossman at google.com>
Date: Tue, 25 Mar 2025 06:28:14 +0000
Subject: [PATCH] [llvm-exegesis] Error Out If Perf Counter is Not Fully
 Enabled

Perf counters can be multiplexed if there are too many that need to be
scheduled on a core at the same time (and they exceed the available
PMUs). Other processes (especially system ones in certain environments,
not commonly on Desktop Linux from what I've seen) can also interfere.
This will impact the measurement fidelity as the counter is not actually
counting cycles/uops the entire time. This patch makes it so that we
error out in these cases so the user gets a visible indication things
have gone wrong rather than things failing silently.
---
 llvm/tools/llvm-exegesis/lib/Error.cpp      | 10 ++++++++++
 llvm/tools/llvm-exegesis/lib/Error.h        | 13 +++++++++++++
 llvm/tools/llvm-exegesis/lib/PerfHelper.cpp | 17 +++++++++++++----
 3 files changed, 36 insertions(+), 4 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/Error.cpp b/llvm/tools/llvm-exegesis/lib/Error.cpp
index 2eee3f2f54c59..2908df25ddb1a 100644
--- a/llvm/tools/llvm-exegesis/lib/Error.cpp
+++ b/llvm/tools/llvm-exegesis/lib/Error.cpp
@@ -49,5 +49,15 @@ void SnippetSignal::log(raw_ostream &OS) const {
 #endif // LLVM_ON_UNIX
 }
 
+char PerfCounterNotFullyEnabled::ID;
+
+std::error_code PerfCounterNotFullyEnabled::convertToErrorCode() const {
+  return inconvertibleErrorCode();
+}
+
+void PerfCounterNotFullyEnabled::log(raw_ostream &OS) const {
+  OS << "The perf counter was not scheduled on the CPU the entire time.";
+}
+
 } // namespace exegesis
 } // namespace llvm
diff --git a/llvm/tools/llvm-exegesis/lib/Error.h b/llvm/tools/llvm-exegesis/lib/Error.h
index 4a3e48997f24f..4e9bafe8daae1 100644
--- a/llvm/tools/llvm-exegesis/lib/Error.h
+++ b/llvm/tools/llvm-exegesis/lib/Error.h
@@ -76,6 +76,19 @@ class SnippetSignal : public SnippetExecutionFailure {
   int SignalNumber;
 };
 
+// A class representing a case where a perf counter was only partially
+// scheduled, most likely due to perf counter contention.
+class PerfCounterNotFullyEnabled
+    : public ErrorInfo<PerfCounterNotFullyEnabled> {
+public:
+  static char ID;
+  PerfCounterNotFullyEnabled() {}
+
+  void log(raw_ostream &OS) const override;
+
+  std::error_code convertToErrorCode() const override;
+};
+
 } // namespace exegesis
 } // namespace llvm
 
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
index 3f3288ceb1e4f..b79f3688c4f21 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
@@ -7,6 +7,7 @@
 //===----------------------------------------------------------------------===//
 
 #include "PerfHelper.h"
+#include "Error.h"
 #include "llvm/Config/config.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/Error.h"
@@ -117,6 +118,8 @@ void ConfiguredEvent::initRealEvent(const pid_t ProcessID, const int GroupFD) {
   const int CPU = -1;
   const uint32_t Flags = 0;
   perf_event_attr AttrCopy = *Event.attribute();
+  AttrCopy.read_format =
+      PERF_FORMAT_TOTAL_TIME_ENABLED | PERF_FORMAT_TOTAL_TIME_RUNNING;
   FileDescriptor = perf_event_open(&AttrCopy, ProcessID, CPU, GroupFD, Flags);
   if (FileDescriptor == -1) {
     errs() << "Unable to open event. ERRNO: " << strerror(errno)
@@ -132,15 +135,21 @@ void ConfiguredEvent::initRealEvent(const pid_t ProcessID, const int GroupFD) {
 
 Expected<SmallVector<int64_t>>
 ConfiguredEvent::readOrError(StringRef /*unused*/) const {
-  int64_t Count = 0;
-  ssize_t ReadSize = ::read(FileDescriptor, &Count, sizeof(Count));
+  int64_t EventInfo[3] = {0, 0, 0};
+  ssize_t ReadSize = ::read(FileDescriptor, &EventInfo, sizeof(EventInfo));
 
-  if (ReadSize != sizeof(Count))
+  if (ReadSize != sizeof(EventInfo))
     return make_error<StringError>("Failed to read event counter",
                                    errc::io_error);
 
+  int64_t EventTimeEnabled = EventInfo[1];
+  int64_t EventTimeRunning = EventInfo[2];
+  if (EventTimeEnabled != EventTimeRunning) {
+    return make_error<PerfCounterNotFullyEnabled>();
+  }
+
   SmallVector<int64_t, 1> Result;
-  Result.push_back(Count);
+  Result.push_back(EventInfo[0]);
   return Result;
 }