[PATCH] D50209: cmake: Explicitely specify benchmark data

[Michael Kruse via Phabricator via llvm-commits]

Meinersbur added a comment.

In https://reviews.llvm.org/D50209#1186520, @MatzeB wrote:

> Some notes:
>
> - I is only for the benchmarks shipping with the test-suite yet. The things in `External` will need some additional changes (they will keep working as is, just won't consistenly have their data in the builddir)
> - The copying grows the builddir from ~900MB to 1GB for me. If this is a concern then we could add a mode that creates symlinks when the test-suite is running locally; however I wasn't convinced yet it is worth having two modes just for 100MB savings in the builddir.


I am indeed concerned. The SPEC (2017) benchmark data can be >100MB per benchmark. In addition, I am adding the build dir to a git repository to see what the differences are (some file types ignored, eg object files). Having the input data in the build dir would blow up that repository, if these are symlinks.

I don't want to block this patch because of my particular setup, but it means that I need something more intelligent than some file extensions in the `.gitignore` of the build dir.



================
Comment at: cmake/modules/TestSuite.cmake:9-25
+function(llvm_test_data target)
+  cmake_parse_arguments(_LTDARGS "" "SOURCE_DIR" "" ${ARGN})
+  set(SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
+  if(_LTDARGS_SOURCE_DIR)
+    set(SOURCE_DIR ${_LTDARGS_SOURCE_DIR})
+  endif()
+  foreach(file ${_LTDARGS_UNPARSED_ARGUMENTS})
----------------
Could we work towards a 'one call=one benchmark specification' approach? Such as

```
llvm_test(progname
  SOURCES ...
  CFLAGS ...
  WORKDIR ...
  RUN_OPTIONS ...
  RUN_DATA ...
  ...
```

I always found the different functions/macros that have to be called in a specific order and pass state between each other (`TESTSCRIPT`) more complex than necessary.



================
Comment at: cmake/modules/TestSuite.cmake:16
+  foreach(file ${_LTDARGS_UNPARSED_ARGUMENTS})
+    llvm_copy(${target} $<TARGET_FILE_DIR:${target}>/${file} ${SOURCE_DIR}/${file})
+  endforeach()
----------------
In case of SPEC 2017, a working directory is created where (when necessary) the run data is copied to and the program writes its data to. Some benchmarks construct the input file path from a sibling dir name (`run_${runtype}`) to `argv[0]` or the current cwd.


================
Comment at: cmake/modules/TestSuite.cmake:20
+
+function(llvm_test_data_dir target)
+  set(SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
----------------
Could `llvm_test_data` automatically determine whether the argument is a directory/file?


Repository:
  rT test-suite

https://reviews.llvm.org/D50209