[PATCH] Add benchmarking-only mode to the test suite

Sun May 11 16:39:51 PDT 2014

Hello everyone,

To follow-up on last week's thread on adding a benchmarking-only mode to the test suite, which excludes those programs unlikely to use useful as benchmarks, here's a patch which provides an implementation. When BENCHMARKING_ONLY is defined, the following programs should be excluded:

MultiSource/Applications/Burg/burg
MultiSource/Applications/treecc/treecc
MultiSource/Benchmarks/MiBench/office-ispell/office-ispell
MultiSource/Benchmarks/MiBench/security-blowfish/security-blowfish
MultiSource/Benchmarks/Prolangs-C/allroots/allroots
MultiSource/Benchmarks/Prolangs-C/archie-client/archie
MultiSource/Benchmarks/Prolangs-C/assembler/assembler
MultiSource/Benchmarks/Prolangs-C/compiler/compiler
MultiSource/Benchmarks/Prolangs-C++/deriv1/deriv1
MultiSource/Benchmarks/Prolangs-C++/deriv2/deriv2
MultiSource/Benchmarks/Prolangs-C++/family/family
MultiSource/Benchmarks/Prolangs-C/fixoutput/fixoutput
MultiSource/Benchmarks/Prolangs-C/football/football
MultiSource/Benchmarks/Prolangs-C++/fsm/fsm
MultiSource/Benchmarks/Prolangs-C++/garage/garage
MultiSource/Benchmarks/Prolangs-C++/NP/np
MultiSource/Benchmarks/Prolangs-C++/objects/objects
MultiSource/Benchmarks/Prolangs-C++/office/office
MultiSource/Benchmarks/Prolangs-C/plot2fig/plot2fig
MultiSource/Benchmarks/Prolangs-C++/shapes/shapes
MultiSource/Benchmarks/Prolangs-C/simulator/simulator
MultiSource/Benchmarks/Prolangs-C/TimberWolfMC/timberwolfmc
MultiSource/Benchmarks/Prolangs-C++/trees/trees
MultiSource/Benchmarks/Prolangs-C/unix-smail/unix-smail
MultiSource/Benchmarks/Prolangs-C/unix-tbl/unix-tbl
MultiSource/Benchmarks/Prolangs-C++/vcirc/vcirc
SingleSource/Benchmarks/McGill/exptree
SingleSource/Benchmarks/Shootout-C++/hello
SingleSource/Benchmarks/Shootout-C++/reversefile
SingleSource/Benchmarks/Shootout-C++/spellcheck
SingleSource/Benchmarks/Shootout-C++/sumcol
SingleSource/Benchmarks/Shootout-C++/wc
SingleSource/Benchmarks/Shootout-C++/wordfreq
SingleSource/Benchmarks/Shootout/hello
SingleSource/Regression/C++/2003-05-14-array-init
SingleSource/Regression/C++/2003-05-14-expr_stmt
SingleSource/Regression/C/2003-05-14-initialize-string
SingleSource/Regression/C/2003-05-21-BitfieldHandling
SingleSource/Regression/C/2003-05-21-UnionBitfields
SingleSource/Regression/C/2003-05-21-UnionTest
SingleSource/Regression/C/2003-05-22-LocalTypeTest
SingleSource/Regression/C/2003-05-22-VarSizeArray
SingleSource/Regression/C/2003-05-23-TransparentUnion
SingleSource/Regression/C++/2003-06-08-BaseType
SingleSource/Regression/C++/2003-06-08-VirtualFunctions
SingleSource/Regression/C++/2003-06-13-Crasher
SingleSource/Regression/C/2003-06-16-InvalidInitializer
SingleSource/Regression/C/2003-06-16-VolatileError
SingleSource/Regression/C++/2003-08-20-EnumSizeProblem
SingleSource/Regression/C++/2003-09-29-NonPODsByValue
SingleSource/Regression/C/2003-10-12-GlobalVarInitializers
SingleSource/Regression/C/2004-02-03-AggregateCopy
SingleSource/Regression/C/2004-03-15-IndirectGoto
SingleSource/Regression/C/2005-05-06-LongLongSignedShift
SingleSource/Regression/C/2008-01-07-LongDouble
SingleSource/Regression/C++/2008-01-29-ParamAliasesReturn
SingleSource/Regression/C++/2011-03-28-Bitfield
SingleSource/Regression/C/badidx
SingleSource/Regression/C/bigstack
SingleSource/Regression/C++/BuiltinTypeInfo
SingleSource/Regression/C/callargs
SingleSource/Regression/C/casts
SingleSource/Regression/C/compare
SingleSource/Regression/C/ConstructorDestructorAttributes
SingleSource/Regression/C/DuffsDevice
SingleSource/Regression/C++/EH/class_hierarchy
SingleSource/Regression/C++/EH/ConditionalExpr
SingleSource/Regression/C++/EH/ctor_dtor_count
SingleSource/Regression/C++/EH/ctor_dtor_count-2
SingleSource/Regression/C++/EH/dead_try_block
SingleSource/Regression/C++/EH/exception_spec_test
SingleSource/Regression/C++/EH/function_try_block
SingleSource/Regression/C++/EH/inlined_cleanup
SingleSource/Regression/C++/EH/recursive-throw
SingleSource/Regression/C++/EH/simple_rethrow
SingleSource/Regression/C++/EH/simple_throw
SingleSource/Regression/C++/EH/throw_rethrow_test
SingleSource/Regression/C++/fixups
SingleSource/Regression/C++/global_ctor
SingleSource/Regression/C/globalrefs
SingleSource/Regression/C++/global_type
SingleSource/Regression/C++/ofstream_ctor
SingleSource/Regression/C/pointer_arithmetic
SingleSource/Regression/C++/pointer_member
SingleSource/Regression/C++/pointer_method
SingleSource/Regression/C++/pointer_method2
SingleSource/Regression/C/PR10189
SingleSource/Regression/C/PR1386
SingleSource/Regression/C/PR491
SingleSource/Regression/C/PR640
SingleSource/Regression/C++/short_circuit_dtor
SingleSource/Regression/C/sumarray
SingleSource/Regression/C/sumarraymalloc
SingleSource/Regression/C/testtrace
SingleSource/UnitTests/2002-04-17-PrintfChar
SingleSource/UnitTests/2002-05-02-ArgumentTest
SingleSource/UnitTests/2002-05-02-CastTest
SingleSource/UnitTests/2002-05-02-CastTest1
SingleSource/UnitTests/2002-05-02-CastTest2
SingleSource/UnitTests/2002-05-02-CastTest3
SingleSource/UnitTests/2002-05-02-ManyArguments
SingleSource/UnitTests/2002-05-03-NotTest
SingleSource/UnitTests/2002-05-19-DivTest
SingleSource/UnitTests/2002-08-02-CastTest
SingleSource/UnitTests/2002-08-02-CastTest2
SingleSource/UnitTests/2002-08-19-CodegenBug
SingleSource/UnitTests/2002-10-09-ArrayResolution
SingleSource/UnitTests/2002-10-12-StructureArgs
SingleSource/UnitTests/2002-10-12-StructureArgsSimple
SingleSource/UnitTests/2002-10-13-BadLoad
SingleSource/UnitTests/2002-12-13-MishaTest
SingleSource/UnitTests/2003-04-22-Switch
SingleSource/UnitTests/2003-05-02-DependentPHI
SingleSource/UnitTests/2003-05-07-VarArgs
SingleSource/UnitTests/2003-05-12-MinIntProblem
SingleSource/UnitTests/2003-05-14-AtExit
SingleSource/UnitTests/2003-05-26-Shorts
SingleSource/UnitTests/2003-05-31-CastToBool
SingleSource/UnitTests/2003-05-31-LongShifts
SingleSource/UnitTests/2003-07-06-IntOverflow
SingleSource/UnitTests/2003-07-08-BitOpsTest
SingleSource/UnitTests/2003-07-09-LoadShorts
SingleSource/UnitTests/2003-07-09-SignedArgs
SingleSource/UnitTests/2003-07-10-SignConversions
SingleSource/UnitTests/2003-08-05-CastFPToUint
SingleSource/UnitTests/2003-08-11-VaListArg
SingleSource/UnitTests/2003-08-20-FoldBug
SingleSource/UnitTests/2003-09-18-BitFieldTest
SingleSource/UnitTests/2003-10-13-SwitchTest
SingleSource/UnitTests/2003-10-29-ScalarReplBug
SingleSource/UnitTests/2004-02-02-NegativeZero
SingleSource/UnitTests/2004-06-20-StaticBitfieldInit
SingleSource/UnitTests/2004-11-28-GlobalBoolLayout
SingleSource/UnitTests/2005-05-11-Popcount-ffs-fls
SingleSource/UnitTests/2005-05-12-Int64ToFP
SingleSource/UnitTests/2005-05-13-SDivTwo
SingleSource/UnitTests/2005-07-17-INT-To-FP
SingleSource/UnitTests/2005-11-29-LongSwitch
SingleSource/UnitTests/2006-01-29-SimpleIndirectCall
SingleSource/UnitTests/2006-02-04-DivRem
SingleSource/UnitTests/2006-12-01-float_varg
SingleSource/UnitTests/2006-12-04-DynAllocAndRestore
SingleSource/UnitTests/2006-12-07-Compare64BitConstant
SingleSource/UnitTests/2006-12-11-LoadConstants
SingleSource/UnitTests/2007-01-04-KNR-Args
SingleSource/UnitTests/2007-03-02-VaCopy
SingleSource/UnitTests/2007-04-25-weak
SingleSource/UnitTests/2008-04-18-LoopBug
SingleSource/UnitTests/2008-04-20-LoopBug2
SingleSource/UnitTests/2008-07-13-InlineSetjmp
SingleSource/UnitTests/2009-04-16-BitfieldInitialization
SingleSource/UnitTests/2009-12-07-StructReturn
SingleSource/UnitTests/2010-05-24-BitfieldTest
SingleSource/UnitTests/AtomicOps
SingleSource/UnitTests/block-byref-cxxobj-test
SingleSource/UnitTests/block-byref-test
SingleSource/UnitTests/block-call-r7674133
SingleSource/UnitTests/block-copied-in-cxxobj
SingleSource/UnitTests/block-copied-in-cxxobj-1
SingleSource/UnitTests/blockstret
SingleSource/UnitTests/byval-alignment
SingleSource/UnitTests/conditional-gnu-ext
SingleSource/UnitTests/conditional-gnu-ext-cxx
SingleSource/UnitTests/DefaultInitDynArrays
SingleSource/UnitTests/FloatPrecision
SingleSource/UnitTests/initp1
SingleSource/UnitTests/member-function-pointers
SingleSource/UnitTests/printargs
SingleSource/UnitTests/SignlessTypes/cast2
SingleSource/UnitTests/SignlessTypes/cast-bug
SingleSource/UnitTests/SignlessTypes/ccc
SingleSource/UnitTests/SignlessTypes/div
SingleSource/UnitTests/SignlessTypes/factor
SingleSource/UnitTests/SignlessTypes/Large/cast
SingleSource/UnitTests/SignlessTypes/shr
SingleSource/UnitTests/stmtexpr
SingleSource/UnitTests/StructModifyTest
SingleSource/UnitTests/TestLoop
SingleSource/UnitTests/Threads/2010-12-08-tls
SingleSource/UnitTests/Threads/tls
SingleSource/UnitTests/Vector/build
SingleSource/UnitTests/Vector/divides
SingleSource/UnitTests/Vector/sumarray
SingleSource/UnitTests/Vector/sumarray-dbl
SingleSource/UnitTests/vla

SingleSource/UnitTests/Vector/Altivec/2007-01-07-lvsl-lvsr-Regression
SingleSource/UnitTests/Vector/Altivec/alti.sdot
SingleSource/UnitTests/Vector/Altivec/casts
SingleSource/UnitTests/Vector/Altivec/test1

SingleSource/UnitTests/ms_struct-bitfield
SingleSource/UnitTests/ms_struct-bitfield-1
SingleSource/UnitTests/ms_struct-bitfield-init
SingleSource/UnitTests/ms_struct-bitfield-init-1
SingleSource/UnitTests/ms_struct_pack_layout
SingleSource/UnitTests/ms_struct_pack_layout-1

Please review.

Thanks again,
Hal

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ts-benchmarking-only.patch
Type: text/x-patch
Size: 12224 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140511/139cd3de/attachment.bin>