[llvm-commits] [PATCH] Adding the TSVC loop benchmarks to the test suite

[Hal Finkel]

On Thu, 27 Sep 2012 09:34:11 +0200
Tobias Grosser <tobias at grosser.es> wrote:

> On 09/27/2012 03:44 AM, Hal Finkel wrote:
> > Please review the attached patch which adds the TSVC loop
> > benchmarks to the LLVM test suite. These are the benchmarks
> > associated with the paper: Maleki, S., et al. An Evaluation of
> > Vectorizing Compilers. PACT'11.
> 
> Nice.
> 
> > I have adjusted each stage so that the double-precision version
> > takes approximately 5 seconds to run on my machine (the float
> > version should be faster, but is not always -- one of the reasons
> > it will be good have this added). With 18 classes each with double
> > and float variants, and assuming that the time for the native
> > binary is comparable to the test binary, the entire benchmark
> > should take approximately 360 seconds to run. Is that too long?
> 
> I believe that is fine. If they run a lot shorter, the timings will
> be very unstable.
> 
> > Do I need to do anything special with the fcmp tolerances?
> 
> When adding the polybench test cases, the only problem I have seen
> was that NaNs caused problems with textual comparisons as it is
> written differently on OS X and Linux. So better ensure your programs
> don't output NaNs.

I'll check this.

> Otherwise, I am not aware of any floating point special handling in
> the test suite. I think for now, strict floating point semantics are 
> expected. (I am not a test suite expert)

The problem with this is that:
  1. gcc does not run with strict semantics by default on many platforms
  2. We might not want to run with strict semantics either (for
  vectorizing reductions, for example).

On the other hand, it might be easier just to adjust the precision with
which the numbers are printed, than to try and fixup the comparison
afterward.

> 
> Also, you may want to add reference_out files.

Good idea.

Thanks again,
Hal

> 
> Cheers
> Tobi



-- 
Hal Finkel
Postdoctoral Appointee
Leadership Computing Facility
Argonne National Laboratory