<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Incremental testing is useful when you're writing code and need to
answer these questions fast:<br>
<br>
<ul>
<li>Did my recent change fix tests that were previously failing?</li>
<li>Are the tests I just modified passing?</li>
</ul>
<br>
A standard check-clang run takes around 40 seconds on my system, and
needs to be run repeatedly while working on a new feature or bug
fix. Most of that is time spent waiting.<br>
<br>
With the attached patch, lit instead runs failing and recently
modified tests first. This way, I'm likely to get an answer to both
questions in under one second, letting me get straight back to
fixing the issue.<br>
<br>
<b>Workflow</b><b><br>
</b><br>
After fixing the code and running lit again, if there are still
failures at the start I'll usually hit Ctrl-C, try to fix the issue
and repeat until all failures are resolved.<br>
<br>
The feature isn't just for local testing -- on our internal clang
build server, LLVM test failures or fixes often get detected within
10 seconds of a commit thanks to ccache/cmake/ninja together with
this patch.<br>
<br>
<b>Performance<br>
<br>
</b>The additional ordering overhead and cache updates come at a
cost of about 2 seconds on top of a 40 second run.<br>
<br>
So enabling the feature depends on your workflow and whether you're
able to act upon the early results.<br>
<br>
For example, the Buildbot version used on llvm.org doesn't send
failure notifications until the test run is complete so the feature
would only be marginally useful there until Buildbot gets early
notification support.<br>
<b><br>
Implementation notes</b><br>
<br>
lit shell tests and compiled unit tests are both supported. At the
moment, the cache is implemented by scanning and modifying test
source mtimes directly. This hasn't caused trouble but the technique
could be refined in future to store test failures in a separate
cache file if needed.<br>
<br>
<b>Status</b><b><br>
</b><br>
Not yet tested on Windows. Let me know if you find this useful!<br>
<br>
Alp.<br>
<br>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.nuanti.com">http://www.nuanti.com</a>
the browser experts
</pre>
</body>
</html>