[PATCH] lit: Incremental test scheduling

[Alp Toker]

To revisit this, I spent some time to build a database to avoid touching 
the source mtimes but it turned out to be a non-trivial amount of code.

It also lost one of the handy properties where we incidentally order on 
source modification time across multiple build output directories.

Another problem was "garbage collection" where if you run lit on a 
subdirectory, or when tests fail the database continues to grow and 
would need more logic for pruning.

Meanwhile I've had a lot of success on full dependency scanning which 
I'm going to complete and propose a few months into 2014. If that works 
out, it'll comprehensively replace this one so I don't think right now 
it's worth developing this approach beyond my original patch.

A few people have pinged on the patch so it's fair to say the patch is 
being applied and used locally.

For these reasons, I propose landing the original patch tweaked to be 
off by default and enabled with the short flag "-i" noting that source 
mtimes are modified in the help string.

Let me know if you think this is OK to land.

Alp.



On 28/10/2013 02:04, Daniel Dunbar wrote:
> I like the functionality, but I will have to think some more about the 
> implementation.
>
> I have already considered having lit maintain some sort of database of 
> test results. That would be useful for reporting things like flaky 
> tests, reporting performance test changes, and for this feature.
>
> The other thing that would be really nice to open the door for would 
> be if lit could detect which tools have changed and only rerun tests 
> for those tools. Implementing this feature by having lit keep a 
> database of test results might provide a base for that...
>
> I'll think some more about this and get back to you with some 
> more concrete ideas next week.
>
>  - Daniel
>
> On Sunday, October 27, 2013, Andrew Trick wrote:
>
>     I really want this feature. It may not be clean, but would save a
>     lot of time in practice. However...
>
>     On Oct 24, 2013, at 10:35 PM, Alp Toker <alp at nuanti.com
>     <javascript:_e({}, 'cvml', 'alp at nuanti.com');>> wrote:
>
>>     lit shell tests and compiled unit tests are both supported. At
>>     the moment, the cache is implemented by scanning and modifying
>>     test source mtimes directly. This hasn't caused trouble but the
>>     technique could be refined in future to store test failures in a
>>     separate cache file if needed.
>
>     This doesn’t feel right. I don’t think it will personally cause me
>     any grief, but seems like someone or their tools will be confused.
>     If others think it’s ok, I won’t object. I just don’t want to
>     accept the patch until others review this aspect.
>
>     -Andy
>
>
>
> -- 
> - Daniel

-- 
http://www.nuanti.com
the browser experts