[PATCH] D33343: Add some tips on how to benchhmark

[Kristof Beyls via Phabricator via llvm-commits]

kristof.beyls added inline comments.


================
Comment at: docs/Benchmarking.rst:1-3
+==================================
+Benchmarking tips
+==================================
----------------
I think that at the moment, these are mainly tips specifically for how to set up a system to reduce noisiness in benchmark results.
Therefore, I'm wondering if the title should be a bit more specific than "Benchmarking tips". But I also can't immediately come up with a title that's much better...


================
Comment at: docs/Benchmarking.rst:9
+
+For benchmarking a patch we want to be in control of all the possible sources of noise. How to do that is very OS dependent.
+
----------------
nitpick: s/to be in control of all the possible sources of noise/reduce all possible sources of noise as much as possible/?
I think that's slightly better, but don't feel strongly about this change.


================
Comment at: docs/Benchmarking.rst:25
+    mount -t tmpfs -o size=<XX>g none dir_to_mount
+
+Linux
----------------
At http://lnt.llvm.org/quickstart.html#running-tests, there are a few hints under number 3 on how to reduce noise and cope with the remaining noise better when you use LNT for benchmarking. Would it be useful to have a pointer here to there?
Maybe some of the hints there might also make sense to have here?
Or maybe some of them should just be moved here, and there can be a pointer from there to here?


================
Comment at: docs/Benchmarking.rst:29-31
+* Disable address space randomization::
+
+    echo 0 > /proc/sys/kernel/randomize_va_space
----------------
Results will be less noisy with address space randomization turned off, but I continue to think that incorrectly skews experiment results.
If you recommend this, maybe you also need to recommend to align all functions to a reasonably large offset so that small code changes in one function have less of a probability of affecting code layout and the associated performance impact in another function?


https://reviews.llvm.org/D33343