[PATCH] D30488: [LNT] Add a few hints on how to reduce noisiness.

[Kristof Beyls via Phabricator via llvm-commits]

kristof.beyls created this revision.

https://reviews.llvm.org/D30488

Files:
  docs/quickstart.rst


Index: docs/quickstart.rst
===================================================================
--- docs/quickstart.rst
+++ docs/quickstart.rst
@@ -70,6 +70,26 @@
    The ``SANDBOX`` value is a path to where the test suite build products and
    results will be stored (inside a timestamped directory, by default).
 
+#. On most systems, the execution time results will be a bit noisy. There are
+   a range of things you can do to reduce noisiness (with LNT runtest nt
+   command line options when available between brackets):
+
+   * Only build the benchmarks in parallel, but do the actual running of the
+     benchmark code at most one at a time. (``--threads 1 --build-threads 6``).
+   * When running under linux: Make lnt use linux perf to get more accurate
+     timing for short-running benchmarks (``--use-perf=1``)
+   * Pin the running benchmark to a specific core, so the OS doesn't move the
+     benchmark process from core to core. (Under linux:
+     ``--make-param=RUNUNDER=taskset -c 1``)
+   * Only run the programs that are marked as a benchmark; some of the tests
+     in the test-suite are not intended to be used as a benchmark.
+     (``--benchmarking-only``)
+   * Make sure each program gets run multiple times, so that LNT has a higher
+     chance of recognizing which programs are inherently noisy
+     (``--multisample=5``)
+   * Disable frequency scaling / turbo boost. In case of thermal throttling it
+     can skew the results.
+
 
 Viewing Results
 ---------------


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D30488.90127.patch
Type: text/x-patch
Size: 1504 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170301/5f51bd76/attachment.bin>