[test-suite] r251431 - [test-suite] Introduce a new CMake+LIT based harness
Chris Matthews via llvm-commits
llvm-commits at lists.llvm.org
Tue Oct 27 12:55:36 PDT 2015
Before things start getting ripped out, we need LNT to get support for this.
> On Oct 27, 2015, at 12:09 PM, David Blaikie via llvm-commits <llvm-commits at lists.llvm.org> wrote:
>
>
>
> On Tue, Oct 27, 2015 at 11:15 AM, James Molloy via llvm-commits <llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>> wrote:
> Author: jamesm
> Date: Tue Oct 27 13:15:37 2015
> New Revision: 251431
>
> URL: http://llvm.org/viewvc/llvm-project?rev=251431&view=rev <http://llvm.org/viewvc/llvm-project?rev=251431&view=rev>
> Log:
> [test-suite] Introduce a new CMake+LIT based harness
>
> We have a lot of layers, alternatives, etc, in the test-suite already - do you have a plan/timeline for removing all that cruft once this solution is baked?
>
>
> This introduces a replacement for the Makefile based build system. This replacement build system uses CMake for configuring and compiling the tests, and LIT for running them. LIT will report both compile and execution times as metrics, available by using the '-o' option to output JSON or '-v' (without '-s').
>
> The advantages of this setup are:
> * Use of the same technologies used in LLVM itself means there's more expertise available to debug, fix and extend, and fewer disparate command line flags to remember.
> * A common-off-the-shelf solution means there's much less custom code to maintain, only glue.
> * More readable output by default, more debuggable by far.
> * The ability to run just one test, quickly and easily.
> * The ability to perform test repeats without cleaning and recompiling first.
> * Because the command line is simple, LNT is not required to run a test.
>
> The disadvantages are:
> * Not all of the features in the Makefile system are supported or are going to be. For example, llvm-gcc support, compiling with opt & llc. Clang is stable and expected now - we don't need these workarounds.
>
> This commit introduces the CMake and LIT glue and a python script that can parse all the Makefiles in SingleSource & MultiSource and can generate equivalent CMakeLists.txt. The intention is that this is run once and the resulting files checked in then never used again.
>
> With this script run, the entire test-suite builds and runs. Example command lines:
>
> $ mkdir build; cd build
> $ cmake -DCMAKE_C_COMPILER=/path/to/my/clang .. # CMake finds clang++ and introspects the target OS/arch/endianness
> $ make -j12
> $ llvm-lit -sv .
>
> Ninja also works, but the way the ninja makefiles are generated means there is no way to build all tests in an arbitrary subtree (as LNT's --only-test does). With ninja you can build all tests or a single test.
>
> All feedback welcome!
>
> Added:
> test-suite/trunk/CMakeLists.txt
> test-suite/trunk/cmake/
> test-suite/trunk/cmake/lit-test-template.in <http://lit-test-template.in/>
> test-suite/trunk/cmake/modules/
> test-suite/trunk/cmake/modules/DetectArchitecture.c
> test-suite/trunk/cmake/modules/DetectArchitecture.cmake
> test-suite/trunk/cmake/modules/FindYACC.cmake
> test-suite/trunk/cmake/modules/MakefileFunctions.cmake
> test-suite/trunk/cmake/modules/SingleMultiSource.cmake
> test-suite/trunk/lit.cfg
> test-suite/trunk/tools/CMakeLists.txt
>
> Added: test-suite/trunk/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/CMakeLists.txt?rev=251431&view=auto <http://llvm.org/viewvc/llvm-project/test-suite/trunk/CMakeLists.txt?rev=251431&view=auto>
> ==============================================================================
> --- test-suite/trunk/CMakeLists.txt (added)
> +++ test-suite/trunk/CMakeLists.txt Tue Oct 27 13:15:37 2015
> @@ -0,0 +1,72 @@
> +cmake_minimum_required(VERSION 3.2)
> +
> +# The test-suite is designed to be built in release mode anyway and
> +# falls over unless -DNDEBUG is set.
> +if (NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
> + message(STATUS "No build type selected, defaulting to Release")
> + set(CMAKE_BUILD_TYPE "Release")
> +endif()
> +
> +add_definitions(-DNDEBUG)
> +
> +# Let's make sure lit output is pretty.
> +if(CMAKE_VERSION VERSION_LESS 3.1.20141117)
> + set(cmake_3_2_USES_TERMINAL)
> +else()
> + set(cmake_3_2_USES_TERMINAL USES_TERMINAL)
> +endif()
> +
> +project(test-suite C CXX)
> +
> +# Add path for custom modules
> +set(CMAKE_MODULE_PATH
> + ${CMAKE_MODULE_PATH}
> + "${CMAKE_CURRENT_SOURCE_DIR}/cmake"
> + "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules"
> + )
> +
> +# Sanity check our source directory to make sure that we are not trying to
> +# generate an in-tree build (unless on MSVC_IDE, where it is ok), and to make
> +# sure that we don't have any stray generated files lying around in the tree
> +# (which would end up getting picked up by header search, instead of the correct
> +# versions).
> +if( CMAKE_SOURCE_DIR STREQUAL CMAKE_BINARY_DIR AND NOT MSVC_IDE )
> + message(FATAL_ERROR "In-source builds are not allowed.
> +CMake would overwrite the makefiles distributed with LLVM.
> +Please create a directory and run cmake from there, passing the path
> +to this source directory as the last argument.
> +This process created the file `CMakeCache.txt' and the directory `CMakeFiles'.
> +Please delete them.")
> +endif()
> +
> +include(MakefileFunctions)
> +include(SingleMultiSource)
> +find_package(YACC)
> +find_package(TCL)
> +
> +if(NOT DEFINED TARGET_OS)
> + message(STATUS "Check target operating system - ${CMAKE_SYSTEM_NAME}")
> + set(TARGET_OS ${CMAKE_SYSTEM_NAME})
> +endif()
> +if(NOT DEFINED ARCH)
> + include(DetectArchitecture)
> + detect_architecture(ARCH)
> +endif()
> +if(NOT DEFINED ENDIAN)
> + include(TestBigEndian)
> + test_big_endian(IS_BIGENDIAN)
> + if(IS_BIGENDIAN)
> + set(ENDIAN "big")
> + else()
> + set(ENDIAN "little")
> + endif()
> +endif()
> +
> +add_subdirectory(tools)
> +
> +# Now that the tools have been created, use tools/timeit to time all other compilations.
> +set(CMAKE_C_COMPILE_OBJECT "${CMAKE_BINARY_DIR}/tools/timeit --summary <OBJECT>.time ${CMAKE_C_COMPILE_OBJECT}")
> +set(CMAKE_CXX_COMPILE_OBJECT "${CMAKE_BINARY_DIR}/tools/timeit --summary <OBJECT>.time ${CMAKE_CXX_COMPILE_OBJECT}")
> +
> +add_subdirectory(SingleSource)
> +add_subdirectory(MultiSource)
>
> Added: test-suite/trunk/cmake/lit-test-template.in <http://lit-test-template.in/>
> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/lit-test-template.in?rev=251431&view=auto <http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/lit-test-template.in?rev=251431&view=auto>
> ==============================================================================
> --- test-suite/trunk/cmake/lit-test-template.in <http://lit-test-template.in/> (added)
> +++ test-suite/trunk/cmake/lit-test-template.in <http://lit-test-template.in/> Tue Oct 27 13:15:37 2015
> @@ -0,0 +1,3 @@
> +; RUN: ${RUNUNDER} ${CMAKE_SOURCE_DIR}/RunSafely.sh -t ${TIMEIT} 7200 ${STDIN_FILENAME} %t ${CMAKE_CURRENT_BINARY_DIR}/${exename} ${RUN_OPTIONS}
> +; RUN: ${PROGRAM_OUTPUT_FILTER} %t
> +; RUN: ${DIFFPROG} %t ${REFERENCE_OUTPUT}
>
> Added: test-suite/trunk/cmake/modules/DetectArchitecture.c
> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/DetectArchitecture.c?rev=251431&view=auto <http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/DetectArchitecture.c?rev=251431&view=auto>
> ==============================================================================
> --- test-suite/trunk/cmake/modules/DetectArchitecture.c (added)
> +++ test-suite/trunk/cmake/modules/DetectArchitecture.c Tue Oct 27 13:15:37 2015
> @@ -0,0 +1,23 @@
> +#if defined(__aarch64__)
> +const char *str = "ARCHITECTURE IS AArch64";
> +#elif defined(__arm__)
> +const char *str = "ARCHITECTURE IS ARM";
> +#elif defined(__alpha__)
> +const char *str = "ARCHITECTURE IS Alpha";
> +#elif defined(__mips__)
> +const char *str = "ARCHITECTURE IS Mips";
> +#elif defined(__powerpc__) || defined(__ppc__) || defined(__power__)
> +const char *str = "ARCHITECTURE IS PowerPC";
> +#elif defined(__sparc__)
> +const char *str = "ARCHITECTURE IS Sparc";
> +#elif defined(__xcore__)
> +const char *str = "ARCHITECTURE IS XCore";
> +#elif defined(__i386__) || defined(__x86_64__)
> +const char *str = "ARCHITECTURE IS x86";
> +#endif
> +
> +int main(int argc, char **argv) {
> + int require = str[argc];
> + (void)argv;
> + return require;
> +}
>
> Added: test-suite/trunk/cmake/modules/DetectArchitecture.cmake
> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/DetectArchitecture.cmake?rev=251431&view=auto <http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/DetectArchitecture.cmake?rev=251431&view=auto>
> ==============================================================================
> --- test-suite/trunk/cmake/modules/DetectArchitecture.cmake (added)
> +++ test-suite/trunk/cmake/modules/DetectArchitecture.cmake Tue Oct 27 13:15:37 2015
> @@ -0,0 +1,35 @@
> +##===- DetectArchitecture.cmake -------------------------------------------===##
> +#
> +# Performs a try_compile to determine the architecture of the target.
> +#
> +##===----------------------------------------------------------------------===##
> +
> +macro(detect_architecture variable)
> + try_compile(HAVE_${variable}
> + ${CMAKE_BINARY_DIR}
> + ${CMAKE_SOURCE_DIR}/cmake/modules/DetectArchitecture.c
> + OUTPUT_VARIABLE OUTPUT
> + COPY_FILE ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/DetectArchitecture.bin)
> +
> + if(HAVE_${variable})
> + file(STRINGS ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/DetectArchitecture.bin
> + DETECT_ARCH_STRING LIMIT_COUNT 1 REGEX "ARCHITECTURE IS")
> + if(DETECT_ARCH_STRING)
> + string(REGEX MATCH "[^ ]*$" DETECT_ARCH_MATCH ${DETECT_ARCH_STRING})
> + if(DETECT_ARCH_MATCH)
> + message(STATUS "Check target system architecture: ${DETECT_ARCH_MATCH}")
> + set(${variable} ${DETECT_ARCH_MATCH})
> + else()
> + message(SEND_ERROR "Could not detect target system architecture!")
> + endif()
> + else()
> + message(SEND_ERROR "Could not detect target system architecture!")
> + endif()
> + else()
> + message(STATUS "Determine the system architecture - failed")
> + file(APPEND ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeError.log
> + "Determining the system architecture fialed with the following output:\n${OUTPUT}")
> + set(${variable})
> + endif()
> +
> +endmacro(detect_architecture)
> \ No newline at end of file
>
> Added: test-suite/trunk/cmake/modules/FindYACC.cmake
> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/FindYACC.cmake?rev=251431&view=auto <http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/FindYACC.cmake?rev=251431&view=auto>
> ==============================================================================
> --- test-suite/trunk/cmake/modules/FindYACC.cmake (added)
> +++ test-suite/trunk/cmake/modules/FindYACC.cmake Tue Oct 27 13:15:37 2015
> @@ -0,0 +1,24 @@
> +##===- FindYACC.cmake -----------------------------------------------------===##
> +#
> +# Defines custom rules for building a YACC parser. Based on FindBISON.cmake.
> +#
> +##===----------------------------------------------------------------------===##
> +
> +find_program(YACC_EXECUTABLE yacc DOC "Path to the YACC executable")
> +mark_as_advanced(YACC_EXECUTABLE)
> +
> +if(YACC_EXECUTABLE)
> +
> + macro(add_yacc_parser target_name input_grammar output_c output_h)
> + add_custom_command(OUTPUT ${output_c} ${output_h}
> + COMMAND ${YACC_EXECUTABLE}
> + ARGS -d -o ${output_c} ${input_grammar}
> + DEPENDS ${input_grammar}
> + COMMENT "Building YACC parser ${output_c}"
> + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
> + endmacro(add_yacc_parser)
> +
> +endif(YACC_EXECUTABLE)
> +
> +include(FindPackageHandleStandardArgs)
> +find_package_handle_standard_args(YACC DEFAULT_MSG YACC_EXECUTABLE)
>
> Added: test-suite/trunk/cmake/modules/MakefileFunctions.cmake
> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/MakefileFunctions.cmake?rev=251431&view=auto <http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/MakefileFunctions.cmake?rev=251431&view=auto>
> ==============================================================================
> --- test-suite/trunk/cmake/modules/MakefileFunctions.cmake (added)
> +++ test-suite/trunk/cmake/modules/MakefileFunctions.cmake Tue Oct 27 13:15:37 2015
> @@ -0,0 +1,46 @@
> +##===- MakefileFunctions.cmake --------------------------------------------===##
> +#
> +# Defines functions used within subdirectory CMakeLists.txt, that look a bit
> +# like standard GNU Make functions. These support the auto-generation of
> +# CMakeLists.txt from Makefiles, although any use of these should probably be
> +# looked at and manually translated into more idiomatic CMake.
> +#
> +##===----------------------------------------------------------------------===##
> +
> +# llvm_add_subdirectories - wrapper around add_subdirectory that can take
> +# multiple items.
> +macro(llvm_add_subdirectories)
> + foreach(V ${ARGV})
> + add_subdirectory(${V})
> + endforeach(V)
> +endmacro()
> +
> +# llvm_prepend - equivalent to $(addprefix var,prefix). Modifies var in place.
> +function(llvm_prepend var prefix)
> + set(listVar "")
> + foreach(f ${ARGN})
> + list(APPEND listVar "${prefix}${f}")
> + endforeach(f)
> + set(${var} "${listVar}" PARENT_SCOPE)
> +endfunction()
> +
> +# llvm_filter - equivalent to $(filter needle,haystack). Returns a list containing
> +# all of the items in ${ARGN} (after needle) that match needle.
> +macro(llvm_filter output needle)
> + set(${output})
> + foreach(haystack_item ${ARGN})
> + foreach(needle_item ${needle})
> + if("${haystack_item}" STREQUAL "${needle_item}")
> + list(APPEND ${output} ${haystack_item})
> + endif()
> + endforeach()
> + endforeach()
> +endmacro()
> +
> +# llvm_filter_out - equivalent to $(filter-out needle,haystack). Inverse of
> +# llvm_filter.
> +macro(llvm_filter_out output needle)
> + set(${output} "${ARGN}")
> + llvm_filter(tmp_output ${needle} ${ARGN})
> + list(REMOVE_ITEM ${output} ${tmp_output})
> +endmacro()
>
> Added: test-suite/trunk/cmake/modules/SingleMultiSource.cmake
> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/SingleMultiSource.cmake?rev=251431&view=auto <http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/SingleMultiSource.cmake?rev=251431&view=auto>
> ==============================================================================
> --- test-suite/trunk/cmake/modules/SingleMultiSource.cmake (added)
> +++ test-suite/trunk/cmake/modules/SingleMultiSource.cmake Tue Oct 27 13:15:37 2015
> @@ -0,0 +1,194 @@
> +##===- SingleMultiSource.cmake --------------------------------------------===##
> +#
> +# Defines helpers to add executables and tests. The entry points to this
> +# file are:
> +# `llvm_singlesource()` and
> +# `llvm_multisource()`
> +#
> +# Each is a macro that uses the environment it was called in to determine
> +# what to build and how, and generates a test file that can be given to LIT.
> +# The test file is generated at configure time.
> +#
> +##===----------------------------------------------------------------------===##
> +
> +# get_unique_exe_name - Given a source file name after which a test should be
> +# named, create a unique name for the test. Usually this is just the source file
> +# with the suffix stripped, but in some cases this ends up causing duplicates
> +# so attempt to make each unique (by adding pathname segments until they become
> +# unique).
> +#
> +# FIXME: Swap this with a simpler procedure to just append a numeral
> +set_property(GLOBAL PROPERTY registered_executables)
> +function(get_unique_exe_name new_name main_src)
> + get_property(registered_executables GLOBAL PROPERTY registered_executables)
> +
> + string(REGEX REPLACE ".[cp]+$" "" path ${main_src})
> + string(REGEX REPLACE ".*/" "" name ${path})
> + list(FIND registered_executables ${name} name_idx)
> +
> + if(${name_idx} EQUAL -1)
> + set(${new_name} ${name} PARENT_SCOPE)
> + set_property(GLOBAL APPEND PROPERTY registered_executables ${name})
> + return()
> + endif()
> +
> + # There is a clash. Rename the target. Each time around the loop pull in
> + # a new path component.
> + foreach(n RANGE 1 4)
> + string(REGEX REPLACE ".*/([^/]+/${name})" "\\1" name ${path})
> + string(REGEX REPLACE "/" "-" safe_name ${name})
> +
> + list(FIND registered_executables ${safe_name} name_idx)
> + if(${name_idx} EQUAL -1)
> + set(${new_name} ${safe_name} PARENT_SCOPE)
> + set_property(GLOBAL APPEND PROPERTY registered_executables ${safe_name})
> + return()
> + endif()
> + endforeach()
> + message(FATAL_ERROR "Failed to uniquify executable name!")
> +endfunction()
> +
> +# append_cflags - add flags to the CFLAGS for target.
> +macro(append_cflags target flags)
> + if(NOT "${${flags}}" STREQUAL "")
> + get_target_property(old_cflags ${target} COMPILE_FLAGS)
> + if(${old_cflags} STREQUAL "old_cflags-NOTFOUND")
> + set(old_cflags)
> + endif()
> + string(REPLACE ";" " " s "${old_cflags};${${flags}}")
> + set_target_properties(${target} PROPERTIES COMPILE_FLAGS ${s})
> + endif()
> +endmacro()
> +
> +# append_ldflags - add flags to the LDFLAGS for target.
> +macro(append_ldflags target flags)
> + if(NOT "${${flags}}" STREQUAL "")
> + get_target_property(old_ldflags ${target} LINK_FLAGS)
> + if(${old_ldflags} STREQUAL "old_ldflags-NOTFOUND")
> + set(old_ldflags)
> + endif()
> + string(REPLACE ";" " " s "${old_ldflags};${${flags}}")
> + set_target_properties(${target} PROPERTIES LINK_FLAGS ${s})
> + endif()
> +endmacro()
> +
> +# llvm_add_test - Create a .test driver file suitable for LIT.
> +#
> +# The test template lives in cmake/lit-test-template.in <http://lit-test-template.in/> and is configured by this function.
> +function(llvm_add_test name exename)
> + if(NOT DEFINED STDIN_FILENAME)
> + set(STDIN_FILENAME /dev/null)
> + endif()
> +
> + # Hash if we've been asked to, otherwise just use "touch" as an identity function.
> + if(HASH_PROGRAM_OUTPUT)
> + set(PROGRAM_OUTPUT_FILTER ${CMAKE_SOURCE_DIR}/HashProgramOutput.sh)
> + else()
> + set(PROGRAM_OUTPUT_FILTER touch)
> + endif()
> +
> + # Find the reference output file key name.
> + if(SMALL_PROBLEM_SIZE)
> + set(KEY small)
> + elseif(LARGE_PROBLEM_SIZE)
> + set(KEY large)
> + else()
> + set(KEY)
> + endif()
> +
> + # Pick the best reference output based on "programname.reference_output".
> + if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${ENDIAN}-endian.${KEY})
> + set(REFERENCE_OUTPUT ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${ENDIAN}-endian.${KEY})
> + elseif(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${KEY})
> + set(REFERENCE_OUTPUT ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${KEY})
> + elseif(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${ENDIAN}-endian)
> + set(REFERENCE_OUTPUT ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output.${ENDIAN}-endian)
> + elseif(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output)
> + set(REFERENCE_OUTPUT ${CMAKE_CURRENT_SOURCE_DIR}/${name}.reference_output)
> + else()
> + # Just compare to its own output. This will always succeed, but here's hoping the
> + # test in question uses its exit value to determine status, so it'll be caught by
> + # the previous RUN line.
> + set(REFERENCE_OUTPUT "%t")
> + message("-- No reference output found for test ${name}")
> + endif()
> +
> + # If the program is nondeterministic, don't bother diffing and use "touch" again as an identity.
> + if(DEFINED PROGRAM_IS_NONDETERMINISTIC)
> + set(DIFFPROG touch)
> + else()
> + set(DIFFPROG ${CMAKE_BINARY_DIR}/tools/fpcmp)
> + if(DEFINED FP_TOLERANCE)
> + set(DIFFPROG "${DIFFPROG} -r ${FP_TOLERANCE}")
> + endif()
> + if(DEFINED FP_ABSTOLERANCE)
> + set(DIFFPROG "${DIFFPROG} -a ${FP_ABSTOLERANCE}")
> + endif()
> + endif()
> +
> + if(DEFINED RUN_OPTIONS)
> + # RUN_OPTIONS is a semicolon-separated list. Change it into a whitespace-separated string.
> + string(REPLACE ";" " " RUN_OPTIONS "${RUN_OPTIONS}")
> + endif()
> +
> + # FIXME: Sort out how to call timeit, or timeit.sh.
> + set(TIMEIT ${CMAKE_BINARY_DIR}/tools/timeit)
> + # FIXME: Add runtimelimit support.
> +
> + # Now do the actual configuration.
> + configure_file(${CMAKE_SOURCE_DIR}/cmake/lit-test-template.in <http://lit-test-template.in/>
> + ${CMAKE_CURRENT_BINARY_DIR}/${exename}.test)
> +endfunction()
> +
> +# llvm_singlesource - configure the current directory as a SingleSource subdirectory -
> +# i.e. every file in *.{c,cpp,cc} is treated as its own test.
> +macro(llvm_singlesource)
> + file(GLOB sources *.c *.cpp *.cc)
> + foreach(source ${sources})
> + # Find the pure name of the test
> + string(REGEX REPLACE ".[cp]+$" "" path ${source})
> + string(REGEX REPLACE ".*/" "" name ${path})
> +
> + # Should we skip this?
> + list(FIND PROGRAMS_TO_SKIP ${name} name_idx)
> + if(${name_idx} EQUAL -1)
> + get_unique_exe_name(source_exename ${source})
> + add_executable(${source_exename} ${source})
> + append_cflags(${source_exename} CFLAGS)
> + append_cflags(${source_exename} CPPFLAGS)
> + append_cflags(${source_exename} CXXFLAGS)
> + append_ldflags(${source_exename} LDFLAGS)
> + llvm_add_test(${name} ${source_exename})
> + add_dependencies(${source_exename} timeit fpcmp)
> + endif()
> + endforeach()
> +endmacro()
> +
> +# llvm_multisource - configure the current directory as a MultiSource subdirectory -
> +# i.e. there is one test and it consists of all sources in the directory (or a curated
> +# list, if Source is defined).
> +macro(llvm_multisource)
> + if(DEFINED Source)
> + set(sources ${Source})
> + else()
> + file(GLOB sources *.c *.cpp *.cc)
> + endif()
> + list(LENGTH sources sources_len)
> +
> + if(sources_len GREATER 0 AND DEFINED PROG)
> + # Should we skip this?
> + list(FIND PROGRAMS_TO_SKIP ${PROG} name_idx)
> + if(${name_idx} EQUAL -1)
> + include_directories(${CMAKE_CURRENT_SOURCE_DIR})
> + include_directories(${CMAKE_CURRENT_BINARY_DIR})
> + get_unique_exe_name(source_exename "${PROG}.c")
> + add_executable(${source_exename} ${sources})
> + append_cflags(${source_exename} CFLAGS)
> + append_cflags(${source_exename} CPPFLAGS)
> + append_cflags(${source_exename} CXXFLAGS)
> + append_ldflags(${source_exename} LDFLAGS)
> + llvm_add_test(${PROG} ${source_exename})
> + add_dependencies(${source_exename} timeit fpcmp)
> + endif()
> +endif()
> +endmacro()
>
> Added: test-suite/trunk/lit.cfg
> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/lit.cfg?rev=251431&view=auto <http://llvm.org/viewvc/llvm-project/test-suite/trunk/lit.cfg?rev=251431&view=auto>
> ==============================================================================
> --- test-suite/trunk/lit.cfg (added)
> +++ test-suite/trunk/lit.cfg Tue Oct 27 13:15:37 2015
> @@ -0,0 +1,46 @@
> +import lit.formats
> +import lit
> +import os, glob, re
> +
> +def getUserTimeFromTimeOutput(f):
> + with open(f) as fd:
> + l = [l for l in fd.readlines()
> + if l.startswith('user')]
> + assert len(l) == 1
> +
> + m = re.match(r'user\s+([0-9.]+)', l[0])
> + return float(m.group(1))
> +
> +class TestSuiteTest(lit.formats.ShTest):
> + def __init__(self):
> + lit.formats.ShTest.__init__(self, False)
> +
> + def execute(self, test, litConfig):
> + result = lit.formats.ShTest.execute(self, test, litConfig)
> + basepath = os.path.dirname(test.getFilePath())
> +
> + if not result.code.isFailure:
> + # Collect the timing information.
> + timeglob = os.path.join(basepath, 'Output', '*.time')
> + times = glob.glob(timeglob)
> + assert len(times) == 1
> + time = getUserTimeFromTimeOutput(times[0])
> +
> + result.addMetric('exec_time', lit.Test.toMetricValue(time))
> +
> + # For completeness, attempt to find compile time information too.
> + compile_time = 0.0
> + for path, subdirs, files in os.walk(basepath):
> + for file in files:
> + if file.endswith('.o.time'):
> + compile_time += getUserTimeFromTimeOutput(os.path.join(path, file))
> + result.addMetric('compile_time', lit.Test.toMetricValue(compile_time))
> +
> + return result
> +
> +config.name <http://config.name/> = 'test-suite'
> +
> +config.test_format = TestSuiteTest()
> +config.suffixes = ['.test']
> +config.test_source_root = os.path.dirname(__file__)
> +config.excludes = ['ABI-Testsuite']
>
> Added: test-suite/trunk/tools/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/tools/CMakeLists.txt?rev=251431&view=auto <http://llvm.org/viewvc/llvm-project/test-suite/trunk/tools/CMakeLists.txt?rev=251431&view=auto>
> ==============================================================================
> --- test-suite/trunk/tools/CMakeLists.txt (added)
> +++ test-suite/trunk/tools/CMakeLists.txt Tue Oct 27 13:15:37 2015
> @@ -0,0 +1,5 @@
> +
> +# FIXME: These need to be host-compiled, if we're cross compiling.
> +# FIXME: Replicate Makefile.tools's logic for determining whether to use fpcmp/fpcmp.sh
> +add_executable(fpcmp fpcmp.c)
> +add_executable(timeit timeit.c)
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org <mailto:llvm-commits at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20151027/0fc6977a/attachment.html>
More information about the llvm-commits
mailing list