[llvm] [Exegesis] Add `--dry-run-measurement` (PR #121991)

[via llvm-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-llvm-binary-utilities

Author: Min-Yih Hsu (mshockwave)

<details>
<summary>Changes</summary>

This flag will make llvm-exegesis run everything except the actual snippet execution. It is useful when we want to test some parts of the code between the `assemble-measured-code` and `measure` phase without actually running on native platforms.

---
Full diff: https://github.com/llvm/llvm-project/pull/121991.diff


3 Files Affected:

- (modified) llvm/docs/CommandGuide/llvm-exegesis.rst (+5) 
- (added) llvm/test/tools/llvm-exegesis/dry-run-measurement.test (+11) 
- (modified) llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp (+19-5) 


``````````diff
diff --git a/llvm/docs/CommandGuide/llvm-exegesis.rst b/llvm/docs/CommandGuide/llvm-exegesis.rst
index 8266d891a5e6b1..cd0bce9e2dbcc3 100644
--- a/llvm/docs/CommandGuide/llvm-exegesis.rst
+++ b/llvm/docs/CommandGuide/llvm-exegesis.rst
@@ -449,6 +449,11 @@ OPTIONS
  crash when hardware performance counters are unavailable and for
  debugging :program:`llvm-exegesis` itself.
 
+.. option:: --dry-run-measurement
+  If set, llvm-exegesis runs everything except the actual snippet execution.
+  This is useful if we want to test some part of the code without actually
+  running on native platforms.
+
 .. option:: --execution-mode=[inprocess,subprocess]
 
   This option specifies what execution mode to use. The `inprocess` execution
diff --git a/llvm/test/tools/llvm-exegesis/dry-run-measurement.test b/llvm/test/tools/llvm-exegesis/dry-run-measurement.test
new file mode 100644
index 00000000000000..82857e7998b5e6
--- /dev/null
+++ b/llvm/test/tools/llvm-exegesis/dry-run-measurement.test
@@ -0,0 +1,11 @@
+# RUN: llvm-exegesis --mtriple=riscv64 --mcpu=sifive-p470 --mode=latency --opcode-name=ADD --use-dummy-perf-counters --dry-run-measurement | FileCheck %s
+# REQUIRES: riscv-registered-target
+
+# This test makes sure that llvm-exegesis doesn't execute "cross-compiled" snippets in the presence of
+# --dry-run-measurement. RISC-V was chosen simply because most of the time we run tests on X86 machines.
+
+# Should not contain misleading results.
+# CHECK: measurements:    []
+
+# Should not contain error messages like "snippet crashed while running: Segmentation fault".
+# CHECK: error:           ''
diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index a7771b99e97b1a..9b978c558c1fe7 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -53,6 +53,12 @@
 namespace llvm {
 namespace exegesis {
 
+static cl::opt<bool>
+    DryRunMeasurement("dry-run-measurement",
+                      cl::desc("Run every steps in the measurement phase "
+                               "except executing the snippet."),
+                      cl::init(false), cl::Hidden);
+
 BenchmarkRunner::BenchmarkRunner(const LLVMState &State, Benchmark::ModeE Mode,
                                  BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
                                  ExecutionModeE ExecutionMode,
@@ -140,13 +146,21 @@ class InProcessFunctionExecutorImpl : public BenchmarkRunner::FunctionExecutor {
     Scratch->clear();
     {
       auto PS = ET.withSavedState();
+      // We can't directly capture DryRunMeasurement in the lambda below.
+      bool DryRun = DryRunMeasurement;
       CrashRecoveryContext CRC;
       CrashRecoveryContext::Enable();
-      const bool Crashed = !CRC.RunSafely([this, Counter, ScratchPtr]() {
-        Counter->start();
-        this->Function(ScratchPtr);
-        Counter->stop();
-      });
+      const bool Crashed =
+          !CRC.RunSafely([this, Counter, ScratchPtr, DryRun]() {
+            if (DryRun) {
+              Counter->start();
+              Counter->stop();
+            } else {
+              Counter->start();
+              this->Function(ScratchPtr);
+              Counter->stop();
+            }
+          });
       CrashRecoveryContext::Disable();
       PS.reset();
       if (Crashed) {

``````````

</details>


https://github.com/llvm/llvm-project/pull/121991