[PATCH] Add benchmarking-only mode to the test suite

[Tobias Grosser]

On 23/05/2014 17:36, Yi Kong wrote:
> On 20 May 2014 17:08, Tobias Grosser <tobias at grosser.es> wrote:
>> This means we are obviously breaking those limits. However, this should be
>> rather predictable, no?
> Apparently not.
>
> # started on Fri May 23 15:04:44 2014
>   Performance counter stats for './3mm.simple':
>
>        14099.995114 task-clock (msec)         #    1.000 CPUs utilized
>       4,477,210,010 L1-dcache-load-misses     #  317.533 M/sec
>           [66.66%]
>         171,091,671 L1-dcache-store-misses    #   12.134 M/sec
>           [66.68%]
>           7,280,044 L1-icache-load-misses     #    0.516 M/sec
>           [66.68%]
>       3,215,844,875 LLC-loads                 #  228.074 M/sec
>           [66.71%]
>          17,047,853 LLC-stores                #    1.209 M/sec
>           [66.68%]
>         357,245,416 cache-misses              #   25.337 M/sec
>           [66.63%]
>
>
> # started on Fri May 23 15:05:02 2014
>   Performance counter stats for './3mm.simple':
>
>        10874.834701 task-clock (msec)         #    1.000 CPUs utilized
>       3,907,819,424 L1-dcache-load-misses     #  359.345 M/sec
>           [66.66%]
>          84,628,748 L1-dcache-store-misses    #    7.782 M/sec
>           [66.70%]
>           6,989,338 L1-icache-load-misses     #    0.643 M/sec
>           [66.70%]
>       3,226,399,896 LLC-loads                 #  296.685 M/sec
>           [66.67%]
>          17,862,459 LLC-stores                #    1.643 M/sec
>           [66.66%]
>         183,435,107 cache-misses              #   16.868 M/sec
>           [66.65%]

So what do these numbers tell us? We have a lot more cache misses in the 
first run. however, the ratio (66%) remains identical. Does this mean we 
load values in generally a lot more often? That seems surprising.

> Hal, can you add -DSMALL_DATASET to Makefile or change the array size
> for STANDARD_DATASET to avoid filling up L1d cache?

The interesting optimization problem here is exactly cache behavior. By 
using -DSMALL_DATASET we just avoid it and can measure it any more. That 
would be unfortunate.

Cheers,
Tobias