[LLVMdev] [RFC] Benchmarking subset of the test suite

Nadav Rotem nrotem at apple.com
Sun May 4 16:13:22 PDT 2014 

On May 4, 2014, at 11:37 AM, Chris Lattner <clattner at apple.com> wrote:

> On May 4, 2014, at 5:39 AM, Hal Finkel <hfinkel at anl.gov> wrote:
>> Without further ado, I propose that a test-suite configuration designed for benchmarking exclude the following:
> 
> Unrelatedly to your point, I think we should just outright remove the bulk of the SingleSource/Regression / SingleSource/UnitTests tests.

I agree that we should not use these programs for benchmarking the compiler but I actually like having these tests around for correctness of the compiler.  When developing new optimizations it is really convenient to debug a single file testcase. 

>  They largely date back to before Clang existed and aren’t serving much useful value anymore.
> 
> Any of them that are useful (e.g. the EH tests?) could be saved or moved somewhere else.
> 
> -Chris
> 
> 
>> 
>> SingleSource/Regression/C++/2003-05-14-array-init
>> SingleSource/Regression/C++/2003-05-14-expr_stmt
>> SingleSource/Regression/C/2003-05-14-initialize-string
>> SingleSource/Regression/C/2003-05-21-BitfieldHandling
>> SingleSource/Regression/C/2003-05-21-UnionBitfields
>> SingleSource/Regression/C/2003-05-21-UnionTest
>> SingleSource/Regression/C/2003-05-22-LocalTypeTest
>> SingleSource/Regression/C/2003-05-22-VarSizeArray
>> SingleSource/Regression/C/2003-05-23-TransparentUnion
>> SingleSource/Regression/C++/2003-06-08-BaseType
>> SingleSource/Regression/C++/2003-06-08-VirtualFunctions
>> SingleSource/Regression/C++/2003-06-13-Crasher
>> SingleSource/Regression/C/2003-06-16-InvalidInitializer
>> SingleSource/Regression/C/2003-06-16-VolatileError
>> SingleSource/Regression/C++/2003-08-20-EnumSizeProblem
>> SingleSource/Regression/C++/2003-09-29-NonPODsByValue
>> SingleSource/Regression/C/2003-10-12-GlobalVarInitializers
>> SingleSource/Regression/C/2004-02-03-AggregateCopy
>> SingleSource/Regression/C/2004-03-15-IndirectGoto
>> SingleSource/Regression/C/2005-05-06-LongLongSignedShift
>> SingleSource/Regression/C/2008-01-07-LongDouble
>> SingleSource/Regression/C++/2008-01-29-ParamAliasesReturn
>> SingleSource/Regression/C++/2011-03-28-Bitfield
>> SingleSource/Regression/C/badidx
>> SingleSource/Regression/C/bigstack
>> SingleSource/Regression/C++/BuiltinTypeInfo
>> SingleSource/Regression/C/callargs
>> SingleSource/Regression/C/casts
>> SingleSource/Regression/C/compare
>> SingleSource/Regression/C/ConstructorDestructorAttributes
>> SingleSource/Regression/C/DuffsDevice
>> SingleSource/Regression/C++/EH/class_hierarchy
>> SingleSource/Regression/C++/EH/ConditionalExpr
>> SingleSource/Regression/C++/EH/ctor_dtor_count
>> SingleSource/Regression/C++/EH/ctor_dtor_count-2
>> SingleSource/Regression/C++/EH/dead_try_block
>> SingleSource/Regression/C++/EH/exception_spec_test
>> SingleSource/Regression/C++/EH/function_try_block
>> SingleSource/Regression/C++/EH/inlined_cleanup
>> SingleSource/Regression/C++/EH/recursive-throw
>> SingleSource/Regression/C++/EH/simple_rethrow
>> SingleSource/Regression/C++/EH/simple_throw
>> SingleSource/Regression/C++/EH/throw_rethrow_test
>> SingleSource/Regression/C++/fixups
>> SingleSource/Regression/C++/global_ctor
>> SingleSource/Regression/C/globalrefs
>> SingleSource/Regression/C++/global_type
>> SingleSource/Regression/C++/ofstream_ctor
>> SingleSource/Regression/C/pointer_arithmetic
>> SingleSource/Regression/C++/pointer_member
>> SingleSource/Regression/C++/pointer_method
>> SingleSource/Regression/C++/pointer_method2
>> SingleSource/Regression/C/PR10189
>> SingleSource/Regression/C/PR1386
>> SingleSource/Regression/C/PR491
>> SingleSource/Regression/C/PR640
>> SingleSource/Regression/C++/short_circuit_dtor
>> SingleSource/Regression/C/sumarray
>> SingleSource/Regression/C/sumarraymalloc
>> SingleSource/Regression/C/testtrace
>> SingleSource/UnitTests/2002-04-17-PrintfChar
>> SingleSource/UnitTests/2002-05-02-ArgumentTest
>> SingleSource/UnitTests/2002-05-02-CastTest
>> SingleSource/UnitTests/2002-05-02-CastTest1
>> SingleSource/UnitTests/2002-05-02-CastTest2
>> SingleSource/UnitTests/2002-05-02-CastTest3
>> SingleSource/UnitTests/2002-05-02-ManyArguments
>> SingleSource/UnitTests/2002-05-03-NotTest
>> SingleSource/UnitTests/2002-05-19-DivTest
>> SingleSource/UnitTests/2002-08-02-CastTest
>> SingleSource/UnitTests/2002-08-02-CastTest2
>> SingleSource/UnitTests/2002-08-19-CodegenBug
>> SingleSource/UnitTests/2002-10-09-ArrayResolution
>> SingleSource/UnitTests/2002-10-12-StructureArgs
>> SingleSource/UnitTests/2002-10-12-StructureArgsSimple
>> SingleSource/UnitTests/2002-10-13-BadLoad
>> SingleSource/UnitTests/2002-12-13-MishaTest
>> SingleSource/UnitTests/2003-04-22-Switch
>> SingleSource/UnitTests/2003-05-02-DependentPHI
>> SingleSource/UnitTests/2003-05-07-VarArgs
>> SingleSource/UnitTests/2003-05-12-MinIntProblem
>> SingleSource/UnitTests/2003-05-14-AtExit
>> SingleSource/UnitTests/2003-05-26-Shorts
>> SingleSource/UnitTests/2003-05-31-CastToBool
>> SingleSource/UnitTests/2003-05-31-LongShifts
>> SingleSource/UnitTests/2003-07-06-IntOverflow
>> SingleSource/UnitTests/2003-07-08-BitOpsTest
>> SingleSource/UnitTests/2003-07-09-LoadShorts
>> SingleSource/UnitTests/2003-07-09-SignedArgs
>> SingleSource/UnitTests/2003-07-10-SignConversions
>> SingleSource/UnitTests/2003-08-05-CastFPToUint
>> SingleSource/UnitTests/2003-08-11-VaListArg
>> SingleSource/UnitTests/2003-08-20-FoldBug
>> SingleSource/UnitTests/2003-09-18-BitFieldTest
>> SingleSource/UnitTests/2003-10-13-SwitchTest
>> SingleSource/UnitTests/2003-10-29-ScalarReplBug
>> SingleSource/UnitTests/2004-02-02-NegativeZero
>> SingleSource/UnitTests/2004-06-20-StaticBitfieldInit
>> SingleSource/UnitTests/2004-11-28-GlobalBoolLayout
>> SingleSource/UnitTests/2005-05-11-Popcount-ffs-fls
>> SingleSource/UnitTests/2005-05-12-Int64ToFP
>> SingleSource/UnitTests/2005-05-13-SDivTwo
>> SingleSource/UnitTests/2005-07-17-INT-To-FP
>> SingleSource/UnitTests/2005-11-29-LongSwitch
>> SingleSource/UnitTests/2006-01-29-SimpleIndirectCall
>> SingleSource/UnitTests/2006-02-04-DivRem
>> SingleSource/UnitTests/2006-12-01-float_varg
>> SingleSource/UnitTests/2006-12-04-DynAllocAndRestore
>> SingleSource/UnitTests/2006-12-07-Compare64BitConstant
>> SingleSource/UnitTests/2006-12-11-LoadConstants
>> SingleSource/UnitTests/2007-01-04-KNR-Args
>> SingleSource/UnitTests/2007-03-02-VaCopy
>> SingleSource/UnitTests/2007-04-25-weak
>> SingleSource/UnitTests/2008-04-18-LoopBug
>> SingleSource/UnitTests/2008-04-20-LoopBug2
>> SingleSource/UnitTests/2008-07-13-InlineSetjmp
>> SingleSource/UnitTests/2009-04-16-BitfieldInitialization
>> SingleSource/UnitTests/2009-12-07-StructReturn
>> SingleSource/UnitTests/2010-05-24-BitfieldTest
>> SingleSource/UnitTests/AtomicOps
>> SingleSource/UnitTests/block-byref-cxxobj-test
>> SingleSource/UnitTests/block-byref-test
>> SingleSource/UnitTests/block-call-r7674133
>> SingleSource/UnitTests/block-copied-in-cxxobj
>> SingleSource/UnitTests/block-copied-in-cxxobj-1
>> SingleSource/UnitTests/blockstret
>> SingleSource/UnitTests/byval-alignment
>> SingleSource/UnitTests/conditional-gnu-ext
>> SingleSource/UnitTests/conditional-gnu-ext-cxx
>> SingleSource/UnitTests/DefaultInitDynArrays
>> SingleSource/UnitTests/FloatPrecision
>> SingleSource/UnitTests/initp1
>> SingleSource/UnitTests/member-function-pointers
>> SingleSource/UnitTests/printargs
>> SingleSource/UnitTests/SignlessTypes/cast2
>> SingleSource/UnitTests/SignlessTypes/cast-bug
>> SingleSource/UnitTests/SignlessTypes/ccc
>> SingleSource/UnitTests/SignlessTypes/div
>> SingleSource/UnitTests/SignlessTypes/factor
>> SingleSource/UnitTests/SignlessTypes/Large/cast
>> SingleSource/UnitTests/SignlessTypes/shr
>> SingleSource/UnitTests/stmtexpr
>> SingleSource/UnitTests/StructModifyTest
>> SingleSource/UnitTests/TestLoop
>> SingleSource/UnitTests/Threads/2010-12-08-tls
>> SingleSource/UnitTests/Threads/tls
>> SingleSource/UnitTests/Vector/build
>> SingleSource/UnitTests/Vector/divides
>> SingleSource/UnitTests/Vector/sumarray
>> SingleSource/UnitTests/Vector/sumarray-dbl
>> SingleSource/UnitTests/vla
>> 
>> SingleSource/UnitTests/Vector/Altivec/2007-01-07-lvsl-lvsr-Regression
>> SingleSource/UnitTests/Vector/Altivec/alti.sdot
>> SingleSource/UnitTests/Vector/Altivec/casts
>> SingleSource/UnitTests/Vector/Altivec/test1
>> 
>> SingleSource/UnitTests/ms_struct-bitfield
>> SingleSource/UnitTests/ms_struct-bitfield-1
>> SingleSource/UnitTests/ms_struct-bitfield-init
>> SingleSource/UnitTests/ms_struct-bitfield-init-1
>> SingleSource/UnitTests/ms_struct_pack_layout
>> SingleSource/UnitTests/ms_struct_pack_layout-1
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

More information about the llvm-dev mailing list