<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">I reverted the cmake hackery in r264617 and fixed the compiletime collection by having lit only search for 'branchmarkname.*.o.time' and 'benchmarkname.link.time' in SingleSource subdirectories.</div><div class=""><br class=""></div><div class="">- Matthias</div><br class=""><div><blockquote type="cite" class=""><div class="">On Mar 27, 2016, at 2:31 PM, Matthias Braun via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><br class="Apple-interchange-newline"><br class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class="">On Mar 27, 2016, at 2:28 PM, James Molloy <<a href="mailto:james@jamesmolloy.co.uk" class="">james@jamesmolloy.co.uk</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">It's not just the test file, it's the directory to cd to before invoking "make" - currently to build "foo/bar/baz" we will invoke "cd foo/bar; make baz" assuming foo/bar is a directory and baz is not.<div class=""><br class=""></div></div></div></blockquote>at least with the ninja generator you can do “ninja baz” at the toplevel.</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="">It should be a simple change, the logic just needs to be updated.</div><div class=""><br class=""></div><div class="">James</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Sun, 27 Mar 2016 at 22:07 Matthias Braun via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div class="" style="word-wrap: break-word;"><div class="">Hmm, how about placing all the .test files at the top-level builddir? That way we wouldn’t have that big a disconnect between the ninja target name and the name of the test. The difference would just be a .test suffix...</div></div><div class="" style="word-wrap: break-word;"><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Mar 27, 2016, at 1:53 AM, James Molloy <<a href="mailto:james@jamesmolloy.co.uk" target="_blank" class="">james@jamesmolloy.co.uk</a>> wrote:</div><br class=""><div class="">Hi Matthias,<br class=""><br class="">Thanks for doing this! However I think we also need to update test-suite.py for this change, because I think --only-test handling will now be broken for SingleSource tests.<span class="Apple-converted-space"> </span><br class=""><br class="">James<br class=""><div class="gmail_quote"><div dir="ltr" class="">On Sat, 26 Mar 2016 at 04:11, Matthias Braun via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank" class="">llvm-commits@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">Author: matze<br class="">Date: Fri Mar 25 23:05:50 2016<br class="">New Revision: 264499<br class=""><br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project?rev=264499&view=rev" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project?rev=264499&view=rev</a><br class="">Log:<br class="">cmake/lit: Put SingleSource executables into separate directories<br class=""><br class="">This fixes the problem of compiletimes not getting collected corrected.<br class=""><br class="">Modified:<br class=""> test-suite/trunk/cmake/modules/SingleMultiSource.cmake<br class=""> test-suite/trunk/litsupport/compiletime.py<br class=""><br class="">Modified: test-suite/trunk/cmake/modules/SingleMultiSource.cmake<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/SingleMultiSource.cmake?rev=264499&r1=264498&r2=264499&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/test-suite/trunk/cmake/modules/SingleMultiSource.cmake?rev=264499&r1=264498&r2=264499&view=diff</a><br class="">==============================================================================<br class="">--- test-suite/trunk/cmake/modules/SingleMultiSource.cmake (original)<br class="">+++ test-suite/trunk/cmake/modules/SingleMultiSource.cmake Fri Mar 25 23:05:50 2016<br class="">@@ -136,33 +136,35 @@ macro(test_suite_add_executable name mai<br class=""> list(FIND PROGRAMS_TO_SKIP ${name} name_idx)<br class=""> # Should we skip this?<br class=""> if(${name_idx} EQUAL -1)<br class="">- get_unique_exe_name(source_exename ${mainsource})<br class="">- add_executable(${source_exename} ${ARGN})<br class="">- append_compile_flags(${source_exename} ${CFLAGS})<br class="">- append_compile_flags(${source_exename} ${CPPFLAGS})<br class="">- append_compile_flags(${source_exename} ${CXXFLAGS})<br class="">+ get_unique_exe_name(executable ${mainsource})<br class="">+ add_executable(${executable} ${ARGN})<br class="">+ append_compile_flags(${executable} ${CFLAGS})<br class="">+ append_compile_flags(${executable} ${CPPFLAGS})<br class="">+ append_compile_flags(${executable} ${CXXFLAGS})<br class=""> # Note that we cannot use target_link_libraries() here because that one<br class=""> # only interprets inputs starting with '-' as flags.<br class="">- append_link_flags(${source_exename} ${LDFLAGS})<br class="">+ append_link_flags(${executable} ${LDFLAGS})<br class="">+ set(executable_path ${CMAKE_CURRENT_BINARY_DIR})<br class="">+ if(CMAKE_RUNTIME_OUTPUT_DIRECTORY)<br class="">+ set(executable_path ${executable_path}/${CMAKE_RUNTIME_OUTPUT_DIRECTORY})<br class="">+ endif()<br class="">+ set(executable_path ${executable_path}/${executable})<br class=""> if (TEST_SUITE_PROFILE_USE)<br class="">- append_compile_flags(${source_exename} -fprofile-instr-use=${CMAKE_CURRENT_BINARY_DIR}/${source_exename}.profdata)<br class="">- append_link_flags(${source_exename} -fprofile-instr-use=${CMAKE_CURRENT_BINARY_DIR}/${source_exename}.profdata)<br class="">+ append_compile_flags(${executable} -fprofile-instr-use=${executable_path}.profdata)<br class="">+ append_link_flags(${executable} -fprofile-instr-use=${executable_path}.profdata)<br class=""> endif()<br class=""><br class=""> # Fall back to old style involving RUN_OPTIONS and STDIN_FILENAME if<br class=""> # llvm_test_run() was not called yet.<br class=""> if(NOT TESTSCRIPT)<br class="">- llvm_test_traditional(${CMAKE_CURRENT_BINARY_DIR}/${source_exename}.test<br class="">- ${CMAKE_CURRENT_BINARY_DIR}/${source_exename}<br class="">- ${name})<br class="">+ llvm_test_traditional(${executable_path}.test ${executable_path} ${name})<br class=""> else()<br class="">- llvm_add_test(${CMAKE_CURRENT_BINARY_DIR}/${source_exename}.test<br class="">- ${CMAKE_CURRENT_BINARY_DIR}/${source_exename})<br class="">+ llvm_add_test(${executable_path}.test ${executable_path})<br class=""> endif()<br class=""> if (NOT TEST_SUITE_USE_PERF)<br class="">- add_dependencies(${source_exename} timeit-target)<br class="">+ add_dependencies(${executable} timeit-target)<br class=""> endif()<br class="">- add_dependencies(${source_exename} timeit-host fpcmp-host)<br class="">+ add_dependencies(${executable} timeit-host fpcmp-host)<br class=""> endif()<br class=""> endmacro()<br class=""><br class="">@@ -175,8 +177,14 @@ macro(llvm_singlesource)<br class=""> string(REGEX REPLACE ".[cp]+$" "" path ${source})<br class=""> string(REGEX REPLACE ".*/" "" name ${path})<br class=""><br class="">+ # Setting CMAKE_RUNTIME_OUTPUT_DIRECTORY lets cmake put the files into a<br class="">+ # different subdirectory for each benchmark. This is usefull for<br class="">+ # differentiating which statistics like *.o.time files belongs to which<br class="">+ # benchmark.<br class="">+ set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${name})<br class=""> test_suite_add_executable(${name} ${source} ${source})<br class=""> endforeach()<br class="">+ unset(CMAKE_RUNTIME_OUTPUT_DIRECTORY)<br class=""> endmacro()<br class=""><br class=""> # Configure the current directory as a MultiSource subdirectory - i.e. there is<br class=""><br class="">Modified: test-suite/trunk/litsupport/compiletime.py<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/compiletime.py?rev=264499&r1=264498&r2=264499&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/compiletime.py?rev=264499&r1=264498&r2=264499&view=diff</a><br class="">==============================================================================<br class="">--- test-suite/trunk/litsupport/compiletime.py (original)<br class="">+++ test-suite/trunk/litsupport/compiletime.py Fri Mar 25 23:05:50 2016<br class="">@@ -4,19 +4,26 @@ import timeit<br class=""><br class=""><br class=""> def _getCompileTime(context):<br class="">+ basepath = os.path.dirname(context.test.getFilePath())<br class="">+ dirs = [basepath]<br class="">+ # Single source .o/.o.time files are placed in a different directory.<br class="">+ name = os.path.basename(basepath)<br class="">+ alternate_dir = "%s/../CMakeFiles/%s.dir" % (basepath, name)<br class="">+ dirs.append(alternate_dir)<br class="">+<br class=""> # TODO: This is not correct yet as the directory may contain .o.time files<br class=""> # of multiple benchmarks in the case of SingleSource tests.<br class=""> compile_time = 0.0<br class=""> link_time = 0.0<br class="">- basepath = os.path.dirname(context.test.getFilePath())<br class="">- for path, subdirs, files in os.walk(basepath):<br class="">- for file in files:<br class="">- if file.endswith('.o.time'):<br class="">- fullpath = os.path.join(path, file)<br class="">- compile_time += timeit.getUserTime(fullpath)<br class="">- if file.endswith('.link.time'):<br class="">- fullpath = os.path.join(path, file)<br class="">- link_time += timeit.getUserTime(fullpath)<br class="">+ for dir in dirs:<br class="">+ for path, subdirs, files in os.walk(dir):<br class="">+ for file in files:<br class="">+ if file.endswith('.o.time'):<br class="">+ fullpath = os.path.join(path, file)<br class="">+ compile_time += timeit.getUserTime(fullpath)<br class="">+ if file.endswith('.link.time'):<br class="">+ fullpath = os.path.join(path, file)<br class="">+ link_time += timeit.getUserTime(fullpath)<br class=""> return {<br class=""> 'compile_time': lit.Test.toMetricValue(compile_time),<br class=""> 'link_time': lit.Test.toMetricValue(link_time),<br class=""><br class=""><br class="">_______________________________________________<br class="">llvm-commits mailing list<br class=""><a href="mailto:llvm-commits@lists.llvm.org" target="_blank" class="">llvm-commits@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br class=""></blockquote></div></div></blockquote></div><br class=""></div>_______________________________________________<br class="">llvm-commits mailing list<br class=""><a href="mailto:llvm-commits@lists.llvm.org" target="_blank" class="">llvm-commits@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br class=""></blockquote></div></div></blockquote></div><br class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">llvm-commits mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:llvm-commits@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">llvm-commits@lists.llvm.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></div></blockquote></div><br class=""></body></html>