[all-commits] [llvm/llvm-project] d3202a: [lldb] Parse the dotest output to determine the mo...

[Jonas Devlieghere via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: d3202a5923173bc69767d86e605b012ad09de2a2
      https://github.com/llvm/llvm-project/commit/d3202a5923173bc69767d86e605b012ad09de2a2
  Author: Jonas Devlieghere <jonas at devlieghere.com>
  Date:   2022-06-08 (Wed, 08 Jun 2022)

  Changed paths:
    M lldb/test/API/lldbtest.py

  Log Message:
  -----------
  [lldb] Parse the dotest output to determine the most appropriate result code

Currently we look for keywords in the dotest.py output to determine the
lit result code. This binary approach of a keyword being present works
for PASS and FAIL, where having at least one test pass or fail
respectively results in that exit code. Things are more complicated
for tests that neither passed or failed, but report a combination of
(un)expected failures, skips or unresolved tests.

This patch changes the logic to parse the number of tests with a
particular result from the dotest.py output. For tests that did not PASS
or FAIL, we now report the lit result code for the one that occurred the
most. For example, if we had a test with 3 skips and 4 expected
failures, we report the test as XFAIL.

We're still mapping multiple tests to one result code, so some loss of
information is inevitable.

Differential revision: https://reviews.llvm.org/D127258


  Commit: 37028d3ea7fc9fa7e640c91faa9c347808598dd6
      https://github.com/llvm/llvm-project/commit/37028d3ea7fc9fa7e640c91faa9c347808598dd6
  Author: Jonas Devlieghere <jonas at devlieghere.com>
  Date:   2022-06-08 (Wed, 08 Jun 2022)

  Changed paths:
    M lldb/test/API/api/multithreaded/TestMultithreaded.py

  Log Message:
  -----------
  [lldb] Update TestMultithreaded to report FAIL for a non-zero exit code

A non-zero exit code from the test binary results in a
CalledProcessError. Without catching the exception, that would result in
a error (unresolved test) instead of a failure. This patch fixes that.


Compare: https://github.com/llvm/llvm-project/compare/26053cddb420...37028d3ea7fc