[cfe-dev] Parallelizing clang tests

[Chris Lattner]

On Mar 1, 2008, at 10:54 AM, Gabor Greif wrote:

> I have access to a quad Mac Pro now and have started building clang
> in the llvm tree.
>
> I know that clang's test system is independent of dejagnu (which is a
> good thing).
> While TestRunner.sh is pretty basic it does a good job.
>
> But I expect that the clang tests will multiply soon. So I thought of
> using extra cores to speed up tests.
>
> The idea is this:

This is really cool Gabor.  I'd really like to see this.   One  
additional request: in terms of output, it is nice to be able to get  
output as tests are run.  This lets you know if the test harness has  
hung and lets you see what progress it is making.  However, most tests  
are successful, so seriously verbose output is bad (and potentially  
confusing when -j is used).

How about this approach:

1. by default, a successful test, immediately after it is run, prints  
out a single "." with no newline.
2. by default, a failing test prints out "\n---- foo.c failed ---\n"  
or whatever.
3. When VERBOSE=1 is passed to make, it could print all the tests it  
is running like it does today: print the test name before it is  
executed, which is useful if the test hangs.

This means that we'll get a nice "progress bar" out of dots in the  
common case when everything is passing, but that the order of the dots  
won't matter, and if individual dots get intermixed because of -j, it  
won't matter.

What do you think?

-Chris