[LLVMdev] Use perf tool for more accurate time measuring on Linux

[Tobias Grosser]

On 16/05/2014 18:37, Yi Kong wrote:
> Hi all,
>
> The LLVM benchmarking system produces very noisy results even on quiet
> machines. One of the sources of inaccuracy is the timing tool we are
> using. Because it is a user-space tool, the OS can context switch it
> and we will get an outlier result. Perf stat uses SW_TASK_CLOCK
> counter in kernel to measure time, therefore more accurate. It also
> does not get context switched.
>
> I've implemented a wrapper script over perf stat which mimics the
> behaviour of timeit tool in test suite, so that nothing else needs to
> be modified. The script is not yet feature complete as timeit, but
> enough to run nightly tests.
>
> I carried out experiments on several machines and saw different level
> of improvements. I am no longer seeing outlier results, and MAD is
> considerably lower. The run-by-run changes results over the same
> revision shrank from around 10 to only 2-3. The MAD reduced from
> around 0.01 to 0.003 on a quiet machine.
>
> I've attached the patch and please experiment with it.

I think this is a great idea. I installed perf on all my performance 
trackers. As you already experimented with it on x86, I would be fine 
with just committing the patch. We can adjust it if the results are
not as expected.

Tobias