[lldb-dev] increase timeout for tests?

[Vedant Kumar via lldb-dev]

As a first step, I think there's consensus on increasing the test timeout to ~3x the length of the slowest test we know of. That test appears to be TestDataFormatterObjC, which takes 388 seconds on Davide's machine. So I propose 20 minutes as the timeout value.

Separately, regarding x-failed pexpect()-backed tests, I propose deleting them if they've been x-failed for over a year. That seems like a long enough time to wait for someone to step up and fix them given that they're a real testing/maintenance burden. For any group of to-be-deleted tests, like the 22 lldb-mi tests x-failed in all configurations, I'd file a PR about potentially bringing the tests back. Thoughts?

thanks,
vedant

> On Mar 13, 2018, at 11:52 AM, Davide Italiano <dccitaliano at gmail.com> wrote:
> 
> On Tue, Mar 13, 2018 at 11:26 AM, Jim Ingham <jingham at apple.com> wrote:
>> It sounds like we timing out based on the whole test class, not the individual tests?  If you're worried about test failures not hanging up the test suite the you really want to do the latter.
>> 
>> These are all tests that contain 5 or more independent tests.  That's probably why they are taking so long to run.
>> 
>> I don't object to having fairly long backstop timeouts, though I agree with Pavel that we should choose something reasonable based on the slowest running tests just so some single error doesn't cause test runs to just never complete, making analysis harder.
>> 
> 
> Vedant (cc:ed) is going to take a look at this as he's babysitting the
> bots for the week. I'll defer the call to him.