[test-suite] r286986 - Sort the files generated by SPEC benchmarks

[Johannes Doerfert via llvm-commits]

Author: jdoerfert
Date: Tue Nov 15 11:21:14 2016
New Revision: 286986

URL: http://llvm.org/viewvc/llvm-project?rev=286986&view=rev
Log:
Sort the files generated by SPEC benchmarks

The wildcard used to reference all files generated by a SPEC benchmark
does not guarantee an ordered result. Probably depending on the default
shell e.g., with zsh 5.2, some SPEC benchmarks* did mistakenly fail.

* 177.mesa, 181.mcf, 300.twolf, 429.mcf and 450.soplex

Differential Revision: https://reviews.llvm.org/D26679

Modified:
    test-suite/trunk/External/SPEC/Makefile.spec

Modified: test-suite/trunk/External/SPEC/Makefile.spec
URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/External/SPEC/Makefile.spec?rev=286986&r1=286985&r2=286986&view=diff
==============================================================================
--- test-suite/trunk/External/SPEC/Makefile.spec (original)
+++ test-suite/trunk/External/SPEC/Makefile.spec Tue Nov 15 11:21:14 2016
@@ -13,7 +13,7 @@ SPEC_SANDBOX := $(PROGDIR)/External/SPEC
 # Information about testing the program...
 REF_IN_DIR  := $(SPEC_BENCH_DIR)/data/$(RUN_TYPE)/input/
 REF_OUT_DIR := $(SPEC_BENCH_DIR)/data/$(RUN_TYPE)/output/
-LOCAL_OUTPUTS = $(notdir $(wildcard $(REF_OUT_DIR)/*))
+LOCAL_OUTPUTS = $(sort $(notdir $(wildcard $(REF_OUT_DIR)/*)))
 
 # Sometimes a test will output its program name or some other gunk that
 # we don't want to diff against. Tests can override this definition to