[PATCH] D30488: [LNT] Add a few hints on how to reduce noisiness.

[Phabricator via Phabricator via llvm-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rL297002: Add a few hints on how to reduce noisiness. (authored by kbeyls).

Changed prior to commit:
  https://reviews.llvm.org/D30488?vs=90306&id=90655#toc

Repository:
  rL LLVM

https://reviews.llvm.org/D30488

Files:
  lnt/trunk/docs/quickstart.rst


Index: lnt/trunk/docs/quickstart.rst
===================================================================
--- lnt/trunk/docs/quickstart.rst
+++ lnt/trunk/docs/quickstart.rst
@@ -70,6 +70,29 @@
    The ``SANDBOX`` value is a path to where the test suite build products and
    results will be stored (inside a timestamped directory, by default).
 
+#. On most systems, the execution time results will be a bit noisy. There are
+   a range of things you can do to reduce noisiness (with LNT runtest nt
+   command line options when available between brackets):
+
+   * Only build the benchmarks in parallel, but do the actual running of the
+     benchmark code at most one at a time. (``--threads 1 --build-threads 6``).
+     Of course, when you're also interested in the measured compile time,
+     you should also build sequentially. (``--threads 1 --build-threads 1``).
+   * When running under linux: Make lnt use linux perf to get more accurate
+     timing for short-running benchmarks (``--use-perf=1``)
+   * Pin the running benchmark to a specific core, so the OS doesn't move the
+     benchmark process from core to core. (Under linux:
+     ``--make-param="RUNUNDER=taskset -c 1"``)
+   * Only run the programs that are marked as a benchmark; some of the tests
+     in the test-suite are not intended to be used as a benchmark.
+     (``--benchmarking-only``)
+   * Make sure each program gets run multiple times, so that LNT has a higher
+     chance of recognizing which programs are inherently noisy
+     (``--multisample=5``)
+   * Disable frequency scaling / turbo boost. In case of thermal throttling it
+     can skew the results.
+   * Disable as many processes or services as possible on the target system.
+
 
 Viewing Results
 ---------------


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D30488.90655.patch
Type: text/x-patch
Size: 1768 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170306/479ed310/attachment.bin>