[cfe-dev] Fast "make test" for windows

[Argiris Kirtzidis]

Ted Kremenek wrote:
>
> On Jun 11, 2008, at 8:09 PM, Eli Friedman wrote:
>
>>>
>>> make test : 14m 45s
>>> wintest : 45sec
>>
>> Wow, that's a lot... Process creation is relatively slow on Windows,
>> so I'm not especially surprised.  That said, have you tried to figure
>> out what exactly is slow for the regular make test?
>
> Agreed.  To me this sound suspiciously slow.  There is no reason it 
> should take 14m to run 'make test' on a relatively modern machine.
>
> Argiris:  Are the tests being run serially (only one test at a time)?  
> Are there particular tests that take a long time?  Please try "make 
> test VERBOSE=1" if you haven't already.
>
The tests are being run serially and are about equally slow.
Bash scripts are notoriously slow because Windows doesn't have fork() 
functionality, and MSYS/Cygwin emulate it with dreadful speed results.
The test script is not the only one affected, "./configure" on LLVM 
takes 6 minutes.

That said, I found out that my particular setup was slowing things even 
more (I had plugged MSYS bash shell on a different command prompt 
interface than Windows' native one).
The time for "make test" is now, 8m 10sec.

Eli Friedman wrote:
> It might be
> possible to significantly speed up the tests by reorganizing them a
> bit, so that they don't create so many processes (for example, by
> making TestRunner.sh loop over all the tests, and avoiding some
> repeated work inside TestRunner.sh, and maybe being a bit more clever
> about how we run the tests).  That way, we wouldn't have to worry
> about maintaining two versions of the test runner.
>   

By putting all the test runs from each file into one script and 
executing that, it took only 1m 24sec. Can someone try to modify the 
test script to do something like this ?

> Also, if you're running the tests with cygwin, have you tried using
> msys instead?  It might be significantly faster.
>   

I use MSYS. It's a tiny bit faster.

>   
>> It's a little .NET 2.0 console executable, if anybody is interested let
>> me know and I'll clean it up a bit and post it.
>>     
>
> I'd be interested to see the source to see what it's doing
> differently; depending on how it works, it might be possible to adapt
> the Makefile/shell script to do something similar and see a similar
> speedup without a second testrunner.
>   

It's feeding the test runs directly to Windows' native command line, so 
there's no emulation overhead. Also there's some trickery involved into 
modifying the test command into a format that cmd.exe will accept it.


-Argiris