[LLVMdev] FYI: Changing RunSafely.sh to only track user time

[Chris Lattner]

On Apr 18, 2010, at 11:08 PM, John Criswell wrote:

> Daniel Dunbar wrote:
>> I am going to change the LLVM test-suite RunSafely.sh to only track
>> 'user' time, instead of reporting 'user' + 'sys' time as it currently
>> does. This will probably cause a spike in nightly test numbers,
>> although hopefully it will be limited to the smaller tests.
>> 
>> The eventual goal is to report all numbers (independently), so that we
>> can control for noise better. However, until that happens it is better
>> to track the most stable & interesting number.
>> 
> 
> First, you should be aware that the test-suite infrastructure is used by 
> the Automatic Pool Allocation and SAFECode projects for their testing 
> infrastructure.  It is also used by our internal research projects and 
> may be used by other research projects at other universities.  We use 
> this infrastructure for our research, so changes you make can affect us.

You or other people are welcome to genericize this by adding a knob to control it.  However, mainline llvm should be most concerned with itself, not with projects with out of tree work.  This is very similar to an API change.

I don't really expect it, but worse case, if the harness becomes completely unsuitable for you guys, you could just clone all the makefiles in your tree as a 'report'.  This would be insulate you from changes.

-Chris