[LLVMdev] YA Vectorization Benchmark

[Renato Golin]

On 6 November 2012 22:28, Daniel Dunbar <daniel at zuster.org> wrote:
> Is it possible instead to refactor the tests so that each binary corresponds
> to one test? For example, look at how Hal went about integrating TSVC:

It should be possible. I'll have to understand better what the
preamble does to make sure I'm not stripping out important stuff, but
also what to copy to each kernel's initialization.

Also, I don't know how the timing functions perform across platforms.
I'd have to implement a decent enough timing system, platform
independent, to factor out the initialization step.


> Other things that I would *like* before integrating it:
>  - Rip out the CPU ID stuff, this isn't useful and adds messiness.

Absolutely, that is meaningless.


>  - Have the test just produce output that can be compared, instead of
> including its own check routines

I can make it print numbers in order, is that good enough for the
comparison routines?

If I got it right, the tests self-validates the results, so at least
we know it executed correctly in the end. I can make it produce "OK"
and "FAIL" either way with some numbers stating the timing.


>  - Have the tests run for fixed iterations, instead of doing their own
> adaptive run

Yes, that's rubbish. That was needed to compare the results based on
CPU specific features, but we don't need that.


>  - Produce reference output files, so it works with USE_REFERENCE_OUTPUT=1

Is this a simple diff or do you compare the numerical results by value+-stdev?


cheers,
--renato