[Lldb-commits] [lldb] b950c26 - [lldb/Reproducers] Add top-level-target check-lldb-reproducers

[Jonas Devlieghere via lldb-commits]

Author: Jonas Devlieghere
Date: 2020-05-28T10:48:16-07:00
New Revision: b950c261d9c107d2c9366dd7e583f9f1c4f282f3

URL: https://github.com/llvm/llvm-project/commit/b950c261d9c107d2c9366dd7e583f9f1c4f282f3
DIFF: https://github.com/llvm/llvm-project/commit/b950c261d9c107d2c9366dd7e583f9f1c4f282f3.diff

LOG: [lldb/Reproducers] Add top-level-target check-lldb-reproducers

This adds a new target `check-lldb-reproducers` that replaces the old
`check-lldb-repro`. The latter would only run the shell tests, while
`check-lldb-reproducers` includes the API tests as well. The new target
will be used on GreenDragon.

It's still possible to run just the shell tests with reproducers,
although now that requires crafting the lit invocation yourself. The
parameters haven't changed and are the shame for the API and shell
tests:

  --param lldb-run-with-repro=capture
  --param lldb-run-with-repro=replay

This patch also updates the reproducer documentation.

Added: 
    

Modified: 
    lldb/docs/resources/reproducers.rst
    lldb/test/CMakeLists.txt
    lldb/test/Shell/CMakeLists.txt
    lldb/test/Shell/lit.cfg.py

Removed: 
    


################################################################################
diff  --git a/lldb/docs/resources/reproducers.rst b/lldb/docs/resources/reproducers.rst
index 370a7925a9e9..1b109c06d907 100644
--- a/lldb/docs/resources/reproducers.rst
+++ b/lldb/docs/resources/reproducers.rst
@@ -169,15 +169,18 @@ Reproducers are tested in the following ways:
    directory. These tests serve as integration and regression tests for the
    reproducers infrastructure, as well as doing some sanity checking for basic
    debugger functionality.
- - The shell tests can be run against a reproducer replay. The
-   ``check-lldb-repro`` target will run the shell test suite twice. First it
-   runs the test suite and captures a reproducer for every lldb invocation and
-   saves it to a known location based off lldb's arguments and  working
-   directory. Then it runs the test suite again, this time replaying the
-   reproducers. Certain tests do not fit this paradigm (for example test that
-   check the output of the binary being debugged) and are skipped by marking
-   them as unsupported by adding ``UNSUPPORTED: lldb-repro`` to the top of the
-   test.
+ - The API and shell tests can be run against a replayed reproducer. The
+   ``check-lldb-reproducers`` target will run the API and shell test suite
+   twice: first running the test normally while capturing a reproducer and then
+   a second time using the replayed session as the test input. For the shell
+   tests this use a little shim (``lldb-repro``) that uses the arguments and
+   current working directory to transparently generate or replay a reproducer.
+   For the API tests an extra argument with the reproducer path is passed to
+   ``dotest.py`` which initializes the debugger in the appropriate mode.
+   Certain tests do not fit this paradigm (for example test that check the
+   output of the binary being debugged) and are skipped by marking them as
+   unsupported by adding ``UNSUPPORTED: lldb-repro`` to the top of the shell
+   test or adding the ``skipIfReproducer`` decorator for the API tests.
 
 Additional testing is possible:
 

diff  --git a/lldb/test/CMakeLists.txt b/lldb/test/CMakeLists.txt
index 3cad416f923a..96931cbaeaf9 100644
--- a/lldb/test/CMakeLists.txt
+++ b/lldb/test/CMakeLists.txt
@@ -192,6 +192,27 @@ add_dependencies(check-lldb lldb-test-deps)
 set_target_properties(check-lldb PROPERTIES FOLDER "lldb misc")
 add_dependencies(check-lldb check-lldb-lit)
 
+# Add a lit test suite that runs the API & shell test while capturing a
+# reproducer.
+add_lit_testsuite(check-lldb-reproducers-capture
+  "Running lldb test suite with reproducer capture"
+  ${CMAKE_CURRENT_BINARY_DIR}/API
+  ${CMAKE_CURRENT_BINARY_DIR}/Shell
+  PARAMS "lldb-run-with-repro=capture"
+  EXCLUDE_FROM_CHECK_ALL
+  DEPENDS lldb-test-deps)
+
+# Add a lit test suite that runs the API & shell test by replaying a
+# reproducer.
+add_lit_testsuite(check-lldb-reproducers
+  "Running lldb test suite with reproducer replay"
+  ${CMAKE_CURRENT_BINARY_DIR}/API
+  ${CMAKE_CURRENT_BINARY_DIR}/Shell
+  PARAMS "lldb-run-with-repro=replay"
+  EXCLUDE_FROM_CHECK_ALL
+  DEPENDS lldb-test-deps)
+add_dependencies(check-lldb-reproducers check-lldb-reproducers-capture)
+
 if(LLDB_BUILT_STANDALONE)
   # This has to happen *AFTER* add_lit_testsuite.
   if (EXISTS ${LLVM_MAIN_SRC_DIR}/utils/llvm-lit)

diff  --git a/lldb/test/Shell/CMakeLists.txt b/lldb/test/Shell/CMakeLists.txt
index 0c78e25d9cee..d203f1e093c7 100644
--- a/lldb/test/Shell/CMakeLists.txt
+++ b/lldb/test/Shell/CMakeLists.txt
@@ -15,20 +15,3 @@ if (CMAKE_GENERATOR STREQUAL "Xcode")
     ${CMAKE_CURRENT_BINARY_DIR}
     DEPENDS lldb-test-deps)
 endif()
-
-# Add a lit test suite that runs the shell test while capturing a reproducer.
-add_lit_testsuite(check-lldb-repro-capture
-  "Running lldb shell test suite with reproducer capture"
-  ${CMAKE_CURRENT_BINARY_DIR}
-  PARAMS "lldb-run-with-repro=capture"
-  EXCLUDE_FROM_CHECK_ALL
-  DEPENDS lldb-test-deps)
-
-# Add a lit test suite that runs the shell test by replaying a reproducer.
-add_lit_testsuite(check-lldb-repro
-  "Running lldb shell test suite with reproducer replay"
-  ${CMAKE_CURRENT_BINARY_DIR}
-  PARAMS "lldb-run-with-repro=replay"
-  EXCLUDE_FROM_CHECK_ALL
-  DEPENDS lldb-test-deps)
-add_dependencies(check-lldb-repro check-lldb-repro-capture)

diff  --git a/lldb/test/Shell/lit.cfg.py b/lldb/test/Shell/lit.cfg.py
index 2e23d9dd0eda..2ee646e3fc7d 100644
--- a/lldb/test/Shell/lit.cfg.py
+++ b/lldb/test/Shell/lit.cfg.py
@@ -49,7 +49,7 @@
 lldb_repro_mode = lit_config.params.get('lldb-run-with-repro', None)
 if lldb_repro_mode:
   config.available_features.add('lldb-repro')
-  lit_config.note("Running Shell test with lldb-repro in {} mode.".format(lldb_repro_mode))
+  lit_config.note("Running Shell tests in {} mode.".format(lldb_repro_mode))
   toolchain.use_lldb_repro_substitutions(config, lldb_repro_mode)
 
 llvm_config.use_default_substitutions()