[PATCH] D16688: [test-suite] A number of cmake configuration fixes for External/SPEC.

[Kristof Beyls via llvm-commits]

kristof.beyls added a comment.

In http://reviews.llvm.org/D16688#342180, @MatzeB wrote:

> I just pushed a r259581 which provides an iostream.h and fstream.h in the benchmark directory that just includes iostream and fstream and removes the -stdlib=libstdc++.


Hi Matthias,

I think I've just found out why we've been seeing differences in our runs.
It turns out that spec have made a number of dot-releases of both SPEC2000 and SPEC2006.
I've been using the latest release, i.e. SPEC2000 v1.3 (see http://spec.org/cpu2000/) and SPEC2006 v1.2 (see http://spec.org/cpu2006/).

The issue for 252.eon using iostream.h and fstream.h has been fixed/worked around in the SPEC2000 v1.3 source code (see http://spec.org/cpu2000/docs/changes_in_V1.3.html, search for 252.eon).
Next to the issue in 252.eon, I also see at least an issue with 403.gcc in SPEC2006: the input data files have been renamed, and therefore the inputs as recorded in the CMakefiles don't exist (see http://spec.org/cpu2006/Docs/changes-in-v1.2.html, search for 403.gcc for details).

So, I think before trying to fix the issues I'm seeing, we should first make a decision on which versions of spec2000 and spec2006 we'd want to support in the CMakefiles.
Would it be acceptable to only support spec2000 v1.3 and spec2006 v1.2 - i.e. the latest releases? If not, we'll probably need a few more conditionals in the CMakfiles, based on the spec release - e.g. for the changed names of the input files for 403.gcc.

What do you think?

Thanks,

Kristof


http://reviews.llvm.org/D16688