[PATCH] D17717: [LNT][test-suite] Implement a new option: --single-result

[Kristof Beyls via llvm-commits]

kristof.beyls added inline comments.

================
Comment at: lnt/tests/test_suite.py:154-158
@@ -153,1 +153,7 @@
                          type='choice', default=[])
+        group.add_option("", "--single-result", dest="single_result",
+                         help=("only execute one test and apply "
+                               "--single-result-predicate to calculate the "
+                               "exit status. Do not submit to a LNT instance "
+                               "or create a JSON report (useful for bisection)"))
+        group.add_option("", "--single-result-predicate",
----------------
The help text doesn't indicate that this argument expects a fully-qualified test name.
Also, it seems like this argument changes test execution - so is it best placed in the "Test Execution" group, or here in the "Output Options" group?

All in all, the help text is a bit long. Maybe it would be better to shorten the help text here and write some more elaborate documentation with an example for this in one of the rst files under docs, so that how to use this becomes part of the LNT documentation?
Should the documentation for the bisecter (which I guess lives in zorg) be updated to have an example making use of these options?

================
Comment at: lnt/tests/test_suite.py:230-232
@@ -215,2 +229,5 @@
 
+        if self.opts.single_result:
+            self.opts.only_test = os.path.dirname(self.opts.single_result)
+                
         opts.cmake = resolve_command_path(opts.cmake)
----------------
Don't we have directories with multiple tests in the same directory? e.g. test-suite/SingleSource/Regression/C
Does this mean that at most a full directory can be specified, not a single test?
Would using this to bisect a single test in a directory with multiple tests still work?


Repository:
  rL LLVM

http://reviews.llvm.org/D17717