[lldb-dev] Changing the way dotest reports passes and failures

[Zachary Turner]

Currently when you run dotest.py, at the end it prints a summary of the
test run.  It prints the number of failures, the total number of tests, and
then for each failing test, it shows you what file failed.

I'm interested in changing this so that for the purposes of reporting, it
treats every function in every file as a separate test.

Imagine the case where you've got 5 python files in your test suite, and
each one has 12 individual tests.  In each one, a single test fails, and
the other 4 pass.  Currently, dotest will make it look like you've got a 0%
pass rate, when in truth you've got a 93% pass rate.

So I would propose changing the output of dotest from this:

FAILED (errors=6, skipped=3)
Ran 377 tests.
Failing Tests (146)
FAIL: LLDB (suite) :: Test-rdar-10449092.py (Windows zturner-win81 8
6.2.9200 AMD64 Intel64 Family 6 Model 45 Stepping 7, GenuineIntel)

To something like this:

FAILED (errors=6, skipped=3)
Ran 1,527 tests.
Failing Tests (356)
FAIL: LLDB (suite) :: Test-rdar-10449092.py (2 tests failed) (Windows
zturner-win81 8 6.2.9200 AMD64 Intel64 Family 6 Model 45 Stepping 7,
GenuineIntel)

Any thoughts on this?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150521/31955795/attachment.html>