[test-suite] r251431 - [test-suite] Introduce a new CMake+LIT based harness
David Blaikie via llvm-commits
llvm-commits at lists.llvm.org
Tue Oct 27 12:56:52 PDT 2015
On Tue, Oct 27, 2015 at 12:55 PM, Chris Matthews <chris.matthews at apple.com>
wrote:
> Before things start getting ripped out, we need LNT to get support for
> this.
>
Sure - my request is for a plan. I think it would be bad if the test-suite
grew more crawling without bound, so I'm inclined to push for those working
to add features/alternatives to have some encouragement/goal of removing
some/all of the cruft.
>
> On Oct 27, 2015, at 12:09 PM, David Blaikie via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>
>
> On Tue, Oct 27, 2015 at 11:15 AM, James Molloy via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> Author: jamesm
>> Date: Tue Oct 27 13:15:37 2015
>> New Revision: 251431
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=251431&view=rev
>> Log:
>> [test-suite] Introduce a new CMake+LIT based harness
>>
>
> We have a lot of layers, alternatives, etc, in the test-suite already - do
> you have a plan/timeline for removing all that cruft once this solution is
> baked?
>
>
>>
>> This introduces a replacement for the Makefile based build system. This
>> replacement build system uses CMake for configuring and compiling the
>> tests, and LIT for running them. LIT will report both compile and execution
>> times as metrics, available by using the '-o' option to output JSON or '-v'
>> (without '-s').
>>
>> The advantages of this setup are:
>> * Use of the same technologies used in LLVM itself means there's more
>> expertise available to debug, fix and extend, and fewer disparate command
>> line flags to remember.
>> * A common-off-the-shelf solution means there's much less custom code
>> to maintain, only glue.
>> * More readable output by default, more debuggable by far.
>> * The ability to run just one test, quickly and easily.
>> * The ability to perform test repeats without cleaning and recompiling
>> first.
>> * Because the command line is simple, LNT is not required to run a
>> test.
>>
>> The disadvantages are:
>> * Not all of the features in the Makefile system are supported or are
>> going to be. For example, llvm-gcc support, compiling with opt & llc. Clang
>> is stable and expected now - we don't need these workarounds.
>>
>> This commit introduces the CMake and LIT glue and a python script that
>> can parse all the Makefiles in SingleSource & MultiSource and can generate
>> equivalent CMakeLists.txt. The intention is that this is run once and the
>> resulting files checked in then never used again.
>>
>> With this script run, the entire test-suite builds and runs. Example
>> command lines:
>>
>> $ mkdir build; cd build
>> $ cmake -DCMAKE_C_COMPILER=/path/to/my/clang .. # CMake finds clang++
>> and introspects the target OS/arch/endianness
>> $ make -j12
>> $ llvm-lit -sv .
>>
>> Ninja also works, but the way the ninja makefiles are generated means
>> there is no way to build all tests in an arbitrary subtree (as LNT's
>> --only-test does). With ninja you can build all tests or a single test.
>>
>> All feedback welcome!
>>
>> Added:
>> test-suite/trunk/CMakeLists.txt
>> test-suite/trunk/cmake/
>> test-suite/trunk/cmake/lit-test-template.in
>> test-suite/trunk/cmake/modules/
>> test-suite/trunk/cmake/modules/DetectArchitecture.c
>> test-suite/trunk/cmake/modules/DetectArchitecture.cmake
>> test-suite/trunk/cmake/modules/FindYACC.cmake
>> test-suite/trunk/cmake/modules/MakefileFunctions.cmake
>> test-suite/trunk/cmake/modules/SingleMultiSource.cmake
>> test-suite/trunk/lit.cfg
>> test-suite/trunk/tools/CMakeLists.txt
>>
>> Added: test-suite/trunk/CMakeLists.txt
>> URL:
>> http://llvm.org/viewvc/llvm-project/test-suite/trunk/CMakeLists.txt?rev=251431&view=auto
>>
>> ==============================================================================
>> --- test-suite/trunk/CMakeLists.txt (added)
>> +++ test-suite/trunk/CMakeLists.txt Tue Oct 27 13:15:37 2015
>> @@ -0,0 +1,72 @@
>> +cmake_minimum_required(VERSION 3.2)
>> +
>> +# The test-suite is designed to be built in release mode anyway and
>> +# falls over unless -DNDEBUG is set.
>> +if (NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
>> + message(STATUS "No build type selected, defaulting to Release")
>> + set(CMAKE_BUILD_TYPE "Release")
>> +endif()
>> +
>> +add_definitions(-DNDEBUG)
>> +
>> +# Let's make sure lit output is pretty.
>> +if(CMAKE_VERSION VERSION_LESS 3.1.20141117)
>> + set(cmake_3_2_USES_TERMINAL)
>> +else()
>> + set(cmake_3_2_USES_TERMINAL USES_TERMINAL)
>> +endif()
>> +
>> +project(test-suite C CXX)
>> +
>> +# Add path for custom modules
>> +set(CMAKE_MODULE_PATH
>> + ${CMAKE_MODULE_PATH}
>> + "${CMAKE_CURRENT_SOURCE_DIR}/cmake"
>> + "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules"
>> + )
>> +
>> +# Sanity check our source directory to make sure that we are not trying
>> to
>> +# generate an in-tree build (unless on MSVC_IDE, where it is ok), and to
>> make
>> +# sure that we don't have any stray generated files lying around in the
>> tree
>> +# (which would end up getting picked up by header search, instead of the
>> correct
>> +# versions).
>> +if( CMAKE_SOURCE_DIR STREQUAL CMAKE_BINARY_DIR AND NOT MSVC_IDE )
>> + message(FATAL_ERROR "In-source builds are not allowed.
>> +CMake would overwrite the makefiles distributed with LLVM.
>> +Please create a directory and run cmake from there, passing the path
>> +to this source directory as the last argument.
>> +This process created the file `CMakeCache.txt' and the directory
>> `CMakeFiles'.
>> +Please delete them.")
>> +endif()
>> +
>> +include(MakefileFunctions)
>> +include(SingleMultiSource)
>> +find_package(YACC)
>> +find_package(TCL)
>> +
>> +if(NOT DEFINED TARGET_OS)
>> + message(STATUS "Check target operating system - ${CMAKE_SYSTEM_NAME}")
>> + set(TARGET_OS ${CMAKE_SYSTEM_NAME})
>> +endif()
>> +if(NOT DEFINED ARCH)
>> + include(DetectArchitecture)
>> + detect_architecture(ARCH)
>> +endif()
>> +if(NOT DEFINED ENDIAN)
>> + include(TestBigEndian)
>> + test_big_endian(IS_BIGENDIAN)
>> + if(IS_BIGENDIAN)
>> + set(ENDIAN "big")
>> + else()
>> + set(ENDIAN "little")
>> + endif()
>> +endif()
>> +
>> +add_subdirectory(tools)
>> +
>> +# Now that the tools have been created, use tools/timeit to time all
>> other compilations.
>> +set(CMAKE_C_COMPILE_OBJECT "${CMAKE_BINARY_DIR}/tools/timeit --summary
>> <OBJECT>.time ${CMAKE_C_COMPILE_OBJECT}")
>> +set(CMAKE_CXX_COMPILE_OBJECT "${CMAKE_BINARY_DIR}/tools/timeit --summary
>> <OBJECT>.time ${CMAKE_CXX_COMPILE_OBJECT}")
>> +
>> +add_subdirectory(SingleSource)
>> +add_subdirectory(MultiSource)
>>
>> Added: test-suite/trunk/cmake/lit-test-template.in
>> URL:
>> http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/lit-test-template.in?rev=251431&view=auto
>>
>> ==============================================================================
>> --- test-suite/trunk/cmake/lit-test-template.in (added)
>> +++ test-suite/trunk/cmake/lit-test-template.in Tue Oct 27 13:15:37 2015
>> @@ -0,0 +1,3 @@
>> +; RUN: ${RUNUNDER} ${CMAKE_SOURCE_DIR}/RunSafely.sh -t ${TIMEIT} 7200
>> ${STDIN_FILENAME} %t ${CMAKE_CURRENT_BINARY_DIR}/${exename} ${RUN_OPTIONS}
>> +; RUN: ${PROGRAM_OUTPUT_FILTER} %t
>> +; RUN: ${DIFFPROG} %t ${REFERENCE_OUTPUT}
>>
>> Added: test-suite/trunk/cmake/modules/DetectArchitecture.c
>> URL:
>> http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/DetectArchitecture.c?rev=251431&view=auto
>>
>> ==============================================================================
>> --- test-suite/trunk/cmake/modules/DetectArchitecture.c (added)
>> +++ test-suite/trunk/cmake/modules/DetectArchitecture.c Tue Oct 27
>> 13:15:37 2015
>> @@ -0,0 +1,23 @@
>> +#if defined(__aarch64__)
>> +const char *str = "ARCHITECTURE IS AArch64";
>> +#elif defined(__arm__)
>> +const char *str = "ARCHITECTURE IS ARM";
>> +#elif defined(__alpha__)
>> +const char *str = "ARCHITECTURE IS Alpha";
>> +#elif defined(__mips__)
>> +const char *str = "ARCHITECTURE IS Mips";
>> +#elif defined(__powerpc__) || defined(__ppc__) || defined(__power__)
>> +const char *str = "ARCHITECTURE IS PowerPC";
>> +#elif defined(__sparc__)
>> +const char *str = "ARCHITECTURE IS Sparc";
>> +#elif defined(__xcore__)
>> +const char *str = "ARCHITECTURE IS XCore";
>> +#elif defined(__i386__) || defined(__x86_64__)
>> +const char *str = "ARCHITECTURE IS x86";
>> +#endif
>> +
>> +int main(int argc, char **argv) {
>> + int require = str[argc];
>> + (void)argv;
>> + return require;
>> +}
>>
>> Added: test-suite/trunk/cmake/modules/DetectArchitecture.cmake
>> URL:
>> http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/DetectArchitecture.cmake?rev=251431&view=auto
>>
>> ==============================================================================
>> --- test-suite/trunk/cmake/modules/DetectArchitecture.cmake (added)
>> +++ test-suite/trunk/cmake/modules/DetectArchitecture.cmake Tue Oct 27
>> 13:15:37 2015
>> @@ -0,0 +1,35 @@
>> +##===- DetectArchitecture.cmake
>> -------------------------------------------===##
>> +#
>> +# Performs a try_compile to determine the architecture of the target.
>> +#
>>
>> +##===----------------------------------------------------------------------===##
>> +
>> +macro(detect_architecture variable)
>> + try_compile(HAVE_${variable}
>> + ${CMAKE_BINARY_DIR}
>> + ${CMAKE_SOURCE_DIR}/cmake/modules/DetectArchitecture.c
>> + OUTPUT_VARIABLE OUTPUT
>> + COPY_FILE
>> ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/DetectArchitecture.bin)
>> +
>> + if(HAVE_${variable})
>> + file(STRINGS
>> ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/DetectArchitecture.bin
>> + DETECT_ARCH_STRING LIMIT_COUNT 1 REGEX "ARCHITECTURE IS")
>> + if(DETECT_ARCH_STRING)
>> + string(REGEX MATCH "[^ ]*$" DETECT_ARCH_MATCH
>> ${DETECT_ARCH_STRING})
>> + if(DETECT_ARCH_MATCH)
>> + message(STATUS "Check target system architecture:
>> ${DETECT_ARCH_MATCH}")
>> + set(${variable} ${DETECT_ARCH_MATCH})
>> + else()
>> + message(SEND_ERROR "Could not detect target system
>> architecture!")
>> + endif()
>> + else()
>> + message(SEND_ERROR "Could not detect target system architecture!")
>> + endif()
>> + else()
>> + message(STATUS "Determine the system architecture - failed")
>> + file(APPEND
>> ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeError.log
>> + "Determining the system architecture fialed with the following
>> output:\n${OUTPUT}")
>> + set(${variable})
>> + endif()
>> +
>> +endmacro(detect_architecture)
>> \ No newline at end of file
>>
>> Added: test-suite/trunk/cmake/modules/FindYACC.cmake
>> URL:
>> http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/FindYACC.cmake?rev=251431&view=auto
>>
>> ==============================================================================
>> --- test-suite/trunk/cmake/modules/FindYACC.cmake (added)
>> +++ test-suite/trunk/cmake/modules/FindYACC.cmake Tue Oct 27 13:15:37 2015
>> @@ -0,0 +1,24 @@
>> +##===- FindYACC.cmake
>> -----------------------------------------------------===##
>> +#
>> +# Defines custom rules for building a YACC parser. Based on
>> FindBISON.cmake.
>> +#
>>
>> +##===----------------------------------------------------------------------===##
>> +
>> +find_program(YACC_EXECUTABLE yacc DOC "Path to the YACC executable")
>> +mark_as_advanced(YACC_EXECUTABLE)
>> +
>> +if(YACC_EXECUTABLE)
>> +
>> + macro(add_yacc_parser target_name input_grammar output_c output_h)
>> + add_custom_command(OUTPUT ${output_c} ${output_h}
>> + COMMAND ${YACC_EXECUTABLE}
>> + ARGS -d -o ${output_c} ${input_grammar}
>> + DEPENDS ${input_grammar}
>> + COMMENT "Building YACC parser ${output_c}"
>> + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
>> + endmacro(add_yacc_parser)
>> +
>> +endif(YACC_EXECUTABLE)
>> +
>> +include(FindPackageHandleStandardArgs)
>> +find_package_handle_standard_args(YACC DEFAULT_MSG YACC_EXECUTABLE)
>>
>> Added: test-suite/trunk/cmake/modules/MakefileFunctions.cmake
>> URL:
>> http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/MakefileFunctions.cmake?rev=251431&view=auto
>>
>> ==============================================================================
>> --- test-suite/trunk/cmake/modules/MakefileFunctions.cmake (added)
>> +++ test-suite/trunk/cmake/modules/MakefileFunctions.cmake Tue Oct 27
>> 13:15:37 2015
>> @@ -0,0 +1,46 @@
>> +##===- MakefileFunctions.cmake
>> --------------------------------------------===##
>> +#
>> +# Defines functions used within subdirectory CMakeLists.txt, that look a
>> bit
>> +# like standard GNU Make functions. These support the auto-generation of
>> +# CMakeLists.txt from Makefiles, although any use of these should
>> probably be
>> +# looked at and manually translated into more idiomatic CMake.
>> +#
>>
>> +##===----------------------------------------------------------------------===##
>> +
>> +# llvm_add_subdirectories - wrapper around add_subdirectory that can take
>> +# multiple items.
>> +macro(llvm_add_subdirectories)
>> + foreach(V ${ARGV})
>> + add_subdirectory(${V})
>> + endforeach(V)
>> +endmacro()
>> +
>> +# llvm_prepend - equivalent to $(addprefix var,prefix). Modifies var in
>> place.
>> +function(llvm_prepend var prefix)
>> + set(listVar "")
>> + foreach(f ${ARGN})
>> + list(APPEND listVar "${prefix}${f}")
>> + endforeach(f)
>> + set(${var} "${listVar}" PARENT_SCOPE)
>> +endfunction()
>> +
>> +# llvm_filter - equivalent to $(filter needle,haystack). Returns a list
>> containing
>> +# all of the items in ${ARGN} (after needle) that match needle.
>> +macro(llvm_filter output needle)
>> + set(${output})
>> + foreach(haystack_item ${ARGN})
>> + foreach(needle_item ${needle})
>> + if("${haystack_item}" STREQUAL "${needle_item}")
>> + list(APPEND ${output} ${haystack_item})
>> + endif()
>> + endforeach()
>> + endforeach()
>> +endmacro()
>> +
>> +# llvm_filter_out - equivalent to $(filter-out needle,haystack). Inverse
>> of
>> +# llvm_filter.
>> +macro(llvm_filter_out output needle)
>> + set(${output} "${ARGN}")
>> + llvm_filter(tmp_output ${needle} ${ARGN})
>> + list(REMOVE_ITEM ${output} ${tmp_output})
>> +endmacro()
>>
>> Added: test-suite/trunk/cmake/modules/SingleMultiSource.cmake
>> URL:
>> http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/SingleMultiSource.cmake?rev=251431&view=auto
>>
>> ==============================================================================
>> --- test-suite/trunk/cmake/modules/SingleMultiSource.cmake (added)
>> +++ test-suite/trunk/cmake/modules/SingleMultiSource.cmake Tue Oct 27
>> 13:15:37 2015
>> @@ -0,0 +1,194 @@
>> +##===- SingleMultiSource.cmake
>> --------------------------------------------===##
>> +#
>> +# Defines helpers to add executables and tests. The entry points to this
>> +# file are:
>> +# `llvm_singlesource()` and
>> +# `llvm_multisource()`
>> +#
>> +# Each is a macro that uses the environment it was called in to determine
>> +# what to build and how, and generates a test file that can be given to
>> LIT.
>> +# The test file is generated at configure time.
>> +#
>>
>> +##===----------------------------------------------------------------------===##
>> +
>> +# get_unique_exe_name - Given a source file name after which a test
>> should be
>> +# named, create a unique name for the test. Usually this is just the
>> source file
>> +# with the suffix stripped, but in some cases this ends up causing
>> duplicates
>> +# so attempt to make each unique (by adding pathname segments until they
>> become
>> +# unique).
>> +#
>> +# FIXME: Swap this with a simpler procedure to just append a numeral
>> +set_property(GLOBAL PROPERTY registered_executables)
>> +function(get_unique_exe_name new_name main_src)
>> + get_property(registered_executables GLOBAL PROPERTY
>> registered_executables)
>> +
>> + string(REGEX REPLACE ".[cp]+$" "" path ${main_src})
>> + string(REGEX REPLACE ".*/" "" name ${path})
>> + list(FIND registered_executables ${name} name_idx)
>> +
>> + if(${name_idx} EQUAL -1)
>> + set(${new_name} ${name} PARENT_SCOPE)
>> + set_property(GLOBAL APPEND PROPERTY registered_executables ${name})
>> + return()
>> + endif()
>> +
>> + # There is a clash. Rename the target. Each time around the loop pull
>> in
>> + # a new path component.
>> + foreach(n RANGE 1 4)
>> + string(REGEX REPLACE ".*/([^/]+/${name})" "\\1" name ${path})
>> + string(REGEX REPLACE "/" "-" safe_name ${name})
>> +
>> + list(FIND registered_executables ${safe_name} name_idx)
>> + if(${name_idx} EQUAL -1)
>> + set(${new_name} ${safe_name} PARENT_SCOPE)
>> + set_property(GLOBAL APPEND PROPERTY registered_executables
>> ${safe_name})
>> + return()
>> + endif()
>> + endforeach()
>> + message(FATAL_ERROR "Failed to uniquify executable name!")
>> +endfunction()
>> +
>> +# append_cflags - add flags to the CFLAGS for target.
>> +macro(append_cflags target flags)
>> + if(NOT "${${flags}}" STREQUAL "")
>> + get_target_property(old_cflags ${target} COMPILE_FLAGS)
>> + if(${old_cflags} STREQUAL "old_cflags-NOTFOUND")
>> + set(old_cflags)
>> + endif()
>> + string(REPLACE ";" " " s "${old_cflags};${${flags}}")
>> + set_target_properties(${target} PROPERTIES COMPILE_FLAGS ${s})
>> + endif()
>> +endmacro()
>> +
>> +# append_ldflags - add flags to the LDFLAGS for target.
>> +macro(append_ldflags target flags)
>> + if(NOT "${${flags}}" STREQUAL "")
>> + get_target_property(old_ldflags ${target} LINK_FLAGS)
>> + if(${old_ldflags} STREQUAL "old_ldflags-NOTFOUND")
>> + set(old_ldflags)
>> + endif()
>> + string(REPLACE ";" " " s "${old_ldflags};${${flags}}")
>> + set_target_properties(${target} PROPERTIES LINK_FLAGS ${s})
>> + endif()
>> +endmacro()
>> +
>> +# llvm_add_test - Create a .test driver file suitable for LIT.
>> +#
>> +# The test template lives in cmake/lit-test-template.in and is
>> configured by this function.
>> +function(llvm_add_test name exename)
>> + if(NOT DEFINED STDIN_FILENAME)
>> + set(STDIN_FILENAME /dev/null)
>> + endif()
>> +
>> + # Hash if we've been asked to, otherwise just use "touch" as an
>> identity function.
>> + if(HASH_PROGRAM_OUTPUT)
>> + set(PROGRAM_OUTPUT_FILTER ${CMAKE_SOURCE_DIR}/HashProgramOutput.sh)
>> + else()
>> + set(PROGRAM_OUTPUT_FILTER touch)
>> + endif()
>> +
>> + # Find the reference output file key name.
>> + if(SMALL_PROBLEM_SIZE)
>> + set(KEY small)
>> + elseif(LARGE_PROBLEM_SIZE)
>> + set(KEY large)
>> + else()
>> + set(KEY)
>> + endif()
>> +
>> + # Pick the best reference output based on
>> "programname.reference_output".
>> + if(EXISTS
>> ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${ENDIAN}-endian.${KEY})
>> + set(REFERENCE_OUTPUT
>> ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${ENDIAN}-endian.${KEY})
>> + elseif(EXISTS
>> ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${KEY})
>> + set(REFERENCE_OUTPUT
>> ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${KEY})
>> + elseif(EXISTS
>> ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${ENDIAN}-endian)
>> + set(REFERENCE_OUTPUT
>> ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${ENDIAN}-endian)
>> + elseif(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output)
>> + set(REFERENCE_OUTPUT
>> ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output)
>> + else()
>> + # Just compare to its own output. This will always succeed, but
>> here's hoping the
>> + # test in question uses its exit value to determine status, so it'll
>> be caught by
>> + # the previous RUN line.
>> + set(REFERENCE_OUTPUT "%t")
>> + message("-- No reference output found for test ${name}")
>> + endif()
>> +
>> + # If the program is nondeterministic, don't bother diffing and use
>> "touch" again as an identity.
>> + if(DEFINED PROGRAM_IS_NONDETERMINISTIC)
>> + set(DIFFPROG touch)
>> + else()
>> + set(DIFFPROG ${CMAKE_BINARY_DIR}/tools/fpcmp)
>> + if(DEFINED FP_TOLERANCE)
>> + set(DIFFPROG "${DIFFPROG} -r ${FP_TOLERANCE}")
>> + endif()
>> + if(DEFINED FP_ABSTOLERANCE)
>> + set(DIFFPROG "${DIFFPROG} -a ${FP_ABSTOLERANCE}")
>> + endif()
>> + endif()
>> +
>> + if(DEFINED RUN_OPTIONS)
>> + # RUN_OPTIONS is a semicolon-separated list. Change it into a
>> whitespace-separated string.
>> + string(REPLACE ";" " " RUN_OPTIONS "${RUN_OPTIONS}")
>> + endif()
>> +
>> + # FIXME: Sort out how to call timeit, or timeit.sh.
>> + set(TIMEIT ${CMAKE_BINARY_DIR}/tools/timeit)
>> + # FIXME: Add runtimelimit support.
>> +
>> + # Now do the actual configuration.
>> + configure_file(${CMAKE_SOURCE_DIR}/cmake/lit-test-template.in
>> + ${CMAKE_CURRENT_BINARY_DIR}/${exename}.test)
>> +endfunction()
>> +
>> +# llvm_singlesource - configure the current directory as a SingleSource
>> subdirectory -
>> +# i.e. every file in *.{c,cpp,cc} is treated as its own test.
>> +macro(llvm_singlesource)
>> + file(GLOB sources *.c *.cpp *.cc)
>> + foreach(source ${sources})
>> + # Find the pure name of the test
>> + string(REGEX REPLACE ".[cp]+$" "" path ${source})
>> + string(REGEX REPLACE ".*/" "" name ${path})
>> +
>> + # Should we skip this?
>> + list(FIND PROGRAMS_TO_SKIP ${name} name_idx)
>> + if(${name_idx} EQUAL -1)
>> + get_unique_exe_name(source_exename ${source})
>> + add_executable(${source_exename} ${source})
>> + append_cflags(${source_exename} CFLAGS)
>> + append_cflags(${source_exename} CPPFLAGS)
>> + append_cflags(${source_exename} CXXFLAGS)
>> + append_ldflags(${source_exename} LDFLAGS)
>> + llvm_add_test(${name} ${source_exename})
>> + add_dependencies(${source_exename} timeit fpcmp)
>> + endif()
>> + endforeach()
>> +endmacro()
>> +
>> +# llvm_multisource - configure the current directory as a MultiSource
>> subdirectory -
>> +# i.e. there is one test and it consists of all sources in the directory
>> (or a curated
>> +# list, if Source is defined).
>> +macro(llvm_multisource)
>> + if(DEFINED Source)
>> + set(sources ${Source})
>> + else()
>> + file(GLOB sources *.c *.cpp *.cc)
>> + endif()
>> + list(LENGTH sources sources_len)
>> +
>> + if(sources_len GREATER 0 AND DEFINED PROG)
>> + # Should we skip this?
>> + list(FIND PROGRAMS_TO_SKIP ${PROG} name_idx)
>> + if(${name_idx} EQUAL -1)
>> + include_directories(${CMAKE_CURRENT_SOURCE_DIR})
>> + include_directories(${CMAKE_CURRENT_BINARY_DIR})
>> + get_unique_exe_name(source_exename "${PROG}.c")
>> + add_executable(${source_exename} ${sources})
>> + append_cflags(${source_exename} CFLAGS)
>> + append_cflags(${source_exename} CPPFLAGS)
>> + append_cflags(${source_exename} CXXFLAGS)
>> + append_ldflags(${source_exename} LDFLAGS)
>> + llvm_add_test(${PROG} ${source_exename})
>> + add_dependencies(${source_exename} timeit fpcmp)
>> + endif()
>> +endif()
>> +endmacro()
>>
>> Added: test-suite/trunk/lit.cfg
>> URL:
>> http://llvm.org/viewvc/llvm-project/test-suite/trunk/lit.cfg?rev=251431&view=auto
>>
>> ==============================================================================
>> --- test-suite/trunk/lit.cfg (added)
>> +++ test-suite/trunk/lit.cfg Tue Oct 27 13:15:37 2015
>> @@ -0,0 +1,46 @@
>> +import lit.formats
>> +import lit
>> +import os, glob, re
>> +
>> +def getUserTimeFromTimeOutput(f):
>> + with open(f) as fd:
>> + l = [l for l in fd.readlines()
>> + if l.startswith('user')]
>> + assert len(l) == 1
>> +
>> + m = re.match(r'user\s+([0-9.]+)', l[0])
>> + return float(m.group(1))
>> +
>> +class TestSuiteTest(lit.formats.ShTest):
>> + def __init__(self):
>> + lit.formats.ShTest.__init__(self, False)
>> +
>> + def execute(self, test, litConfig):
>> + result = lit.formats.ShTest.execute(self, test, litConfig)
>> + basepath = os.path.dirname(test.getFilePath())
>> +
>> + if not result.code.isFailure:
>> + # Collect the timing information.
>> + timeglob = os.path.join(basepath, 'Output', '*.time')
>> + times = glob.glob(timeglob)
>> + assert len(times) == 1
>> + time = getUserTimeFromTimeOutput(times[0])
>> +
>> + result.addMetric('exec_time', lit.Test.toMetricValue(time))
>> +
>> + # For completeness, attempt to find compile time information too.
>> + compile_time = 0.0
>> + for path, subdirs, files in os.walk(basepath):
>> + for file in files:
>> + if file.endswith('.o.time'):
>> + compile_time +=
>> getUserTimeFromTimeOutput(os.path.join(path, file))
>> + result.addMetric('compile_time',
>> lit.Test.toMetricValue(compile_time))
>> +
>> + return result
>> +
>> +config.name = 'test-suite'
>> +
>> +config.test_format = TestSuiteTest()
>> +config.suffixes = ['.test']
>> +config.test_source_root = os.path.dirname(__file__)
>> +config.excludes = ['ABI-Testsuite']
>>
>> Added: test-suite/trunk/tools/CMakeLists.txt
>> URL:
>> http://llvm.org/viewvc/llvm-project/test-suite/trunk/tools/CMakeLists.txt?rev=251431&view=auto
>>
>> ==============================================================================
>> --- test-suite/trunk/tools/CMakeLists.txt (added)
>> +++ test-suite/trunk/tools/CMakeLists.txt Tue Oct 27 13:15:37 2015
>> @@ -0,0 +1,5 @@
>> +
>> +# FIXME: These need to be host-compiled, if we're cross compiling.
>> +# FIXME: Replicate Makefile.tools's logic for determining whether to use
>> fpcmp/fpcmp.sh
>> +add_executable(fpcmp fpcmp.c)
>> +add_executable(timeit timeit.c)
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20151027/4eacae85/attachment.html>
More information about the llvm-commits
mailing list