[Lldb-commits] [lldb] e3a7c77 - [lldb/Lit] Change the lldbtest format to behave more like shell test.

[Vedant Kumar via lldb-commits]

> On Jan 24, 2020, at 4:18 PM, Jonas Devlieghere via lldb-commits <lldb-commits at lists.llvm.org> wrote:
> 
> 
> Author: Jonas Devlieghere
> Date: 2020-01-24T16:17:55-08:00
> New Revision: e3a7c7713cd87e37a95a544373cd21f6f06ab94e
> 
> URL: https://github.com/llvm/llvm-project/commit/e3a7c7713cd87e37a95a544373cd21f6f06ab94e
> DIFF: https://github.com/llvm/llvm-project/commit/e3a7c7713cd87e37a95a544373cd21f6f06ab94e.diff
> 
> LOG: [lldb/Lit] Change the lldbtest format to behave more like shell test.
> 
> The current lldbtest format has a number of shortcomings, all related to
> how we omit information based on why the test fails. For example, a
> successful test would print nothing, even when `-a` is passed to lit.
> It's not up to the test format to decide whether to print something or
> not, that's handled by lit itself. For other test results we would
> sometimes print stdout & stderr, but not always, such as when a timeout
> was reached or we couldn't parse the dotest output.
> 
> This patch changes the lldbtest format and makes it behave more like
> lit. We now always print the dotest invocation, the exit code, the
> output to stdout & stderr. If you're used to dealing with ShTests in
> lit, this will feel all very familiar.
> 
> Differential revision: https://reviews.llvm.org/D73384
> 
> Added: 
> 
> 
> Modified: 
>    lldb/test/API/lldbtest.py
> 
> Removed: 
> 
> 
> 
> ################################################################################
> diff  --git a/lldb/test/API/lldbtest.py b/lldb/test/API/lldbtest.py
> index 349a67f22fb3..864d5ea1df03 100644
> --- a/lldb/test/API/lldbtest.py
> +++ b/lldb/test/API/lldbtest.py
> @@ -86,33 +86,46 @@ def execute(self, test, litConfig):
>                 shutil.copy(python, copied_python)
>             cmd[0] = copied_python
> 
> +        timeoutInfo = None
>         try:
>             out, err, exitCode = lit.util.executeCommand(
>                 cmd,
>                 env=test.config.environment,
>                 timeout=litConfig.maxIndividualTestTime)
>         except lit.util.ExecuteCommandTimeoutException:
> -            return (lit.Test.TIMEOUT, 'Reached timeout of {} seconds'.format(
> -                litConfig.maxIndividualTestTime))
> +            timeoutInfo = 'Reached timeout of {} seconds'.format(
> +                litConfig.maxIndividualTestTime)
> +
> +        output = """Script:\n--\n%s\n--\nExit Code: %d\n""" % (
> +            ' '.join(cmd), exitCode)

Heads up, when a test times out this prints:

  File "/Users/vsk/src/llvm-project-master/lldb/test/API/lldbtest.py", line 100, in execute
    ' '.join(cmd), exitCode)
UnboundLocalError: local variable 'exitCode' referenced before assignment


> +        if timeoutInfo is not None:
> +            output += """Timeout: %s\n""" % (timeoutInfo,)
> +        output += "\n"
> +
> +        if out:
> +            output += """Command Output (stdout):\n--\n%s\n--\n""" % (out,)
> +        if err:
> +            output += """Command Output (stderr):\n--\n%s\n--\n""" % (err,)
> +
> +        if timeoutInfo:
> +            return lit.Test.TIMEOUT, output
> 
>         if exitCode:
>             # Match FAIL but not XFAIL.
>             for line in out.splitlines() + err.splitlines():
>                 if line.startswith('FAIL:'):
> -                    return lit.Test.FAIL, out + err
> +                    return lit.Test.FAIL, output
> 
>             if 'XPASS:' in out or 'XPASS:' in err:
> -                return lit.Test.XPASS, out + err
> +                return lit.Test.XPASS, output
> 
>         has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err
>         has_passing_tests = 'PASS:' in out or 'PASS:' in err
>         if has_unsupported_tests and not has_passing_tests:
> -            return lit.Test.UNSUPPORTED, out + err
> +            return lit.Test.UNSUPPORTED, output
> 
>         passing_test_line = 'RESULT: PASSED'
>         if passing_test_line not in out and passing_test_line not in err:
> -            msg = ('Unable to find %r in dotest output (exit code %d):\n\n%s%s'
> -                   % (passing_test_line, exitCode, out, err))
> -            return lit.Test.UNRESOLVED, msg
> +            return lit.Test.UNRESOLVED, output
> 
> -        return lit.Test.PASS, ''
> +        return lit.Test.PASS, output
> 
> 
> 
> _______________________________________________
> lldb-commits mailing list
> lldb-commits at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits