[lldb-dev] test rerun phase is in

[Todd Fiala via lldb-dev]

I am seeing several failures on the Ubuntu 14.04 testbot, but unfortunately
there are a number of changes that went in at the same time on that build.
The failures I'm seeing are not appearing at all related to the test
running infrastructure.

Anybody with a fast Linux system able to take a look to see what exactly is
failing there?

-Todd

On Mon, Dec 14, 2015 at 1:39 PM, Todd Fiala <todd.fiala at gmail.com> wrote:

> Hi all,
>
> I just put in the single-worker, low-load, follow-up test run pass in
> r255543.  Most of the work for it went in late last week, this just mostly
> flips it on.
>
> The feature works like this:
>
> * First test phase works as before: run all tests using whatever level of
> concurrency is normally used.  (e.g. 8 works on an 8-logical-core box).
>
> * Any timeouts, failures, errors, or anything else that would have caused
> a test failure is eligible for rerun if either (1) it was marked as a
> flakey test via the flakey decorator, or (2) if the --rerun-all-issues
> command line flag is provided.
>
> * After the first test phase, if there are any tests that met rerun
> eligibility that would have caused a test failure, those get run using a
> serial test phase.  Their results will overwrite (i.e. replace) the
> previous result for the given test method.
>
> The net result should be that tests that were load sensitive and
> intermittently fail during the first higher-concurrency test phase should
> (in theory) pass in the second, single worker test phase when the test
> suite is only using a single worker.  This should make the test suite
> generate fewer false positives on test failure notification, which should
> make continuous integration servers (testbots) much more useful in terms of
> generating actionable signals caused by version control changes to the lldb
> or related sources.
>
> Please let me know if you see any issues with this when running the test
> suite using the default output.  I'd like to fix this up ASAP.  And for
> those interested in the implementation, I'm happy to do post-commit
> review/changes as needed to get it in good shape.
>
> I'll be watching the  builders now and will address any issues as I see
> them.
>
> Thanks!
> --
> -Todd
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151214/6d50e1bc/attachment.html>