[lldb-dev] test rerun phase is in

[Todd Fiala via lldb-dev]

I'm having some of these blow up.

In the case of test/lang/c/typedef/Testtypedef.py, it looks like some of
the @expected decorators were changed a bit, and perhaps they are not pound
for pound the same.  For example, this test used to really be marked XFAIL
(via an expectedFailureClang directive), but it looks like the current
marking of compiler="clang" is either not right or not working, since the
test is run on OS X and is treated like it is expected to pass.

I'm drilling into that a bit more, that's just the first of several that
fail with these changes on OS X.

On Mon, Dec 14, 2015 at 3:03 PM, Zachary Turner <zturner at google.com> wrote:

> I've checked in r255567 which fixes a problem pointed out by Siva.  It
> doesn't sound related to in 255542, but looking at those logs I can't
> really tell how my CL would be related.  If r255567 doesn't fix the bots,
> would someone mind helping me briefly?  r255542 seems pretty
> straightforward, so I don't see why it would have an effect here.
>
> On Mon, Dec 14, 2015 at 2:35 PM Todd Fiala <todd.fiala at gmail.com> wrote:
>
>> Ah yes I see.  Thanks, Ying (and Siva!  Saw your comments too).
>>
>> On Mon, Dec 14, 2015 at 2:34 PM, Ying Chen <chying at google.com> wrote:
>>
>>> Seems this is the first build that fails, and it only has one CL 255542
>>> <http://llvm.org/viewvc/llvm-project/?view=rev&revision=255542>.
>>>
>>> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9446
>>> I believe Zachary is looking at that problem.
>>>
>>> On Mon, Dec 14, 2015 at 2:18 PM, Todd Fiala <todd.fiala at gmail.com>
>>> wrote:
>>>
>>>> I am seeing several failures on the Ubuntu 14.04 testbot, but
>>>> unfortunately there are a number of changes that went in at the same time
>>>> on that build.  The failures I'm seeing are not appearing at all related to
>>>> the test running infrastructure.
>>>>
>>>> Anybody with a fast Linux system able to take a look to see what
>>>> exactly is failing there?
>>>>
>>>> -Todd
>>>>
>>>> On Mon, Dec 14, 2015 at 1:39 PM, Todd Fiala <todd.fiala at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I just put in the single-worker, low-load, follow-up test run pass in
>>>>> r255543.  Most of the work for it went in late last week, this just mostly
>>>>> flips it on.
>>>>>
>>>>> The feature works like this:
>>>>>
>>>>> * First test phase works as before: run all tests using whatever level
>>>>> of concurrency is normally used.  (e.g. 8 works on an 8-logical-core box).
>>>>>
>>>>> * Any timeouts, failures, errors, or anything else that would have
>>>>> caused a test failure is eligible for rerun if either (1) it was marked as
>>>>> a flakey test via the flakey decorator, or (2) if the --rerun-all-issues
>>>>> command line flag is provided.
>>>>>
>>>>> * After the first test phase, if there are any tests that met rerun
>>>>> eligibility that would have caused a test failure, those get run using a
>>>>> serial test phase.  Their results will overwrite (i.e. replace) the
>>>>> previous result for the given test method.
>>>>>
>>>>> The net result should be that tests that were load sensitive and
>>>>> intermittently fail during the first higher-concurrency test phase should
>>>>> (in theory) pass in the second, single worker test phase when the test
>>>>> suite is only using a single worker.  This should make the test suite
>>>>> generate fewer false positives on test failure notification, which should
>>>>> make continuous integration servers (testbots) much more useful in terms of
>>>>> generating actionable signals caused by version control changes to the lldb
>>>>> or related sources.
>>>>>
>>>>> Please let me know if you see any issues with this when running the
>>>>> test suite using the default output.  I'd like to fix this up ASAP.  And
>>>>> for those interested in the implementation, I'm happy to do post-commit
>>>>> review/changes as needed to get it in good shape.
>>>>>
>>>>> I'll be watching the  builders now and will address any issues as I
>>>>> see them.
>>>>>
>>>>> Thanks!
>>>>> --
>>>>> -Todd
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> -Todd
>>>>
>>>
>>>
>>
>>
>> --
>> -Todd
>>
>


-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151214/31804a41/attachment.html>