[PATCH] D17717: [LNT][test-suite] Implement a new option: --single-result

[James Molloy via llvm-commits]

Hi Kristof,

Thanks for the review. I've removed most of the docstring from the options
and put a new sub section in the LNT documentation with a worked example.

> Don't we have directories with multiple tests in the same directory?

Thanks for spotting this. This was completely wrong, but so was our
handling of --only-test. I fixed that in r263443 and have committed the
modified version of this patch in r263445.

On Tue, 1 Mar 2016 at 07:41 Kristof Beyls via llvm-commits <
llvm-commits at lists.llvm.org> wrote:

> kristof.beyls added inline comments.
>
> ================
> Comment at: lnt/tests/test_suite.py:154-158
> @@ -153,1 +153,7 @@
>                           type='choice', default=[])
> +        group.add_option("", "--single-result", dest="single_result",
> +                         help=("only execute one test and apply "
> +                               "--single-result-predicate to calculate
> the "
> +                               "exit status. Do not submit to a LNT
> instance "
> +                               "or create a JSON report (useful for
> bisection)"))
> +        group.add_option("", "--single-result-predicate",
> ----------------
> The help text doesn't indicate that this argument expects a
> fully-qualified test name.
> Also, it seems like this argument changes test execution - so is it best
> placed in the "Test Execution" group, or here in the "Output Options" group?
>
> All in all, the help text is a bit long. Maybe it would be better to
> shorten the help text here and write some more elaborate documentation with
> an example for this in one of the rst files under docs, so that how to use
> this becomes part of the LNT documentation?
> Should the documentation for the bisecter (which I guess lives in zorg) be
> updated to have an example making use of these options?
>
> ================
> Comment at: lnt/tests/test_suite.py:230-232
> @@ -215,2 +229,5 @@
>
> +        if self.opts.single_result:
> +            self.opts.only_test = os.path.dirname(self.opts.single_result)
> +
>          opts.cmake = resolve_command_path(opts.cmake)
> ----------------
> Don't we have directories with multiple tests in the same directory? e.g.
> test-suite/SingleSource/Regression/C
> Does this mean that at most a full directory can be specified, not a
> single test?
> Would using this to bisect a single test in a directory with multiple
> tests still work?
>
>
> Repository:
>   rL LLVM
>
> http://reviews.llvm.org/D17717
>
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160314/3016f172/attachment.html>