[PATCH] D21360: test-suite cmake: Detect test subdirectories when running cmake.

[Artem Belevich via llvm-commits]

tra added a comment.

> Is this a good idea?


I could use it to experiment with tests I'm planning to run as part of CUDA build-bot without changing buildbot config every time I want to tweak something. I.e. I could configure buildbot with fixed set of parameters, with TEST_SUITE_SUBDIRS including directory *outside* of the tree checked out by buildbot. I could then change contents of that directory when I want to change what/how I test.

> Is it fine to search for */CMakeLists.txt on the toplevel test-suite directory?


I don't see much point of doing that. For directories that are part of the source tree we already know they are there and can include them explicitly (assuming there are only few of them). External directories would also be explicitly specified via TEST_SUITE_SUBDIRS.

Do you have particular use case for wildcard search?

> Should we also check the builddir for it?


Probably not. Build dir is something that can be nuked at any time -- not a good place for tests, IMO.

> Do we need an extra configuration flag?


For wildcard search? Probably not, as I'm not sure we need wildcard search at all.

> We currently detect subdirectories only when there is no list of subdirs in the CMakeCache yet. This makes the behaviour deterministic across cmake runs but cmake will also miss newly checked test-suites when run a 2nd time in a configured build directory.


My vote would be for deterministic behavior. Test setups can be rather heavy-weight. I would not want to re-check gazillion files/directories  on every run just in case some new test was added there.


================
Comment at: CMakeLists.txt:168-174
@@ +167,9 @@
+if(NOT TEST_SUITE_SUBDIRS)
+  file(GLOB sub_cmakelists */CMakeLists.txt)
+  set(TEST_SUITE_SUBDIRS "")
+  foreach(entry ${sub_cmakelists})
+    get_filename_component(subdir ${entry} DIRECTORY)
+    if(NOT ${subdir} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/tools)
+      list(APPEND TEST_SUITE_SUBDIRS ${subdir})
+    endif()
+  endforeach()
----------------
Assuming that we do end up with wildcard search, instead of explicitly excluding directories that do not contain tests (tools in this case), I'd instead explicitly mark directories that contain tests. For instance we may require each top-level test directory to have a test-description.txt and search for that file, instead of CMakeLists.txt.


Repository:
  rL LLVM

http://reviews.llvm.org/D21360