[lldb-dev] TestRaise.py test_restart_bug flakey stats
Todd Fiala via lldb-dev
lldb-dev at lists.llvm.org
Sat Oct 17 18:18:24 PDT 2015
Nope, no good either when I limit the flakey to DWO.
So perhaps I don't understand how the flakey marking works. I thought it
* run the test.
* If it passes, it goes as a successful test. Then we're done.
* run the test again.
* If it passes, then we're done and mark it a successful test. If it
fails, then mark it an expected failure.
But that's definitely not the behavior I'm seeing, as a flakey marking in
the above scheme should never produce a failing test.
I'll have to revisit the flakey test marking to see what it's really doing
since my understanding is clearly flawed!
On Sat, Oct 17, 2015 at 5:57 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
> Hmm, the flakey behavior may be specific to dwo. Testing it locally as
> unconditionally flaky on Linux is failing on dwarf. All the ones I see
> succeed are dwo. I wouldn't expect a diff there but that seems to be the
> So, the request still stands but I won't be surprised if we find that dwo
> sometimes passes while dwarf doesn't (or at least not enough to get through
> the flakey setting).
> On Sat, Oct 17, 2015 at 4:57 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
>> Hi Tamas,
>> I think you grabbed me stats on failing tests in the past. Can you dig
>> up the failure rate for TestRaise.py's test_restart_bug() variants on
>> Ubuntu 14.04 x86_64? I'd like to mark it as flaky on Linux, since it is
>> passing most of the time over here. But I want to see if that's valid
>> across all Ubuntu 14.04 x86_64. (If it is passing some of the time, I'd
>> prefer marking it flakey so that we don't see unexpected successes).
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the lldb-dev