[lldb-dev] test rerun phase is in

[Todd Fiala via lldb-dev]

Hi all,

I just put in the single-worker, low-load, follow-up test run pass in
r255543.  Most of the work for it went in late last week, this just mostly
flips it on.

The feature works like this:

* First test phase works as before: run all tests using whatever level of
concurrency is normally used.  (e.g. 8 works on an 8-logical-core box).

* Any timeouts, failures, errors, or anything else that would have caused a
test failure is eligible for rerun if either (1) it was marked as a flakey
test via the flakey decorator, or (2) if the --rerun-all-issues command
line flag is provided.

* After the first test phase, if there are any tests that met rerun
eligibility that would have caused a test failure, those get run using a
serial test phase.  Their results will overwrite (i.e. replace) the
previous result for the given test method.

The net result should be that tests that were load sensitive and
intermittently fail during the first higher-concurrency test phase should
(in theory) pass in the second, single worker test phase when the test
suite is only using a single worker.  This should make the test suite
generate fewer false positives on test failure notification, which should
make continuous integration servers (testbots) much more useful in terms of
generating actionable signals caused by version control changes to the lldb
or related sources.

Please let me know if you see any issues with this when running the test
suite using the default output.  I'd like to fix this up ASAP.  And for
those interested in the implementation, I'm happy to do post-commit
review/changes as needed to get it in good shape.

I'll be watching the  builders now and will address any issues as I see
them.

Thanks!
-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151214/c246a215/attachment.html>